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FOREWORD 

Maintainability,  a  characteristic  of  design  and  installation 
and  affected  by  various  personnel  and  logistic  factors,  is  one  of 
many  system  requirements  which  must  be  considered  during  the 
system  engineering  effort.  The  degree  of  maintainability  achieved 
depends  upon  the  requirements  imposed  and  management  en^hasis  on 
maintainability.  This  standard  defines  a  carefully  planned  program 
to  be  implemented  for  verification,  demonstration  and  evaluation  of 
maintainabi  lity . 

The  purpose  of  this  standard  is  to  establish  uniform  procedures, 
test  methods,  and  requirements  for  verification,  demonstration,  and 
evaluation  of  the  achievement  of  specified  maintainability  require¬ 
ments  and  for  assessment  of  the  impact  of  planned  logistic  si5)port. 

This  standard  is  applicable  to  all  Department  of  Defense 
procurements  which  require  a  maintainability  verification/demonstra¬ 
tion/evaluation  of  maintainability  requirements. 
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1.1  Purpose.  This  standard  provides  procedures  and  test  methods  for 
verification,  demonstration,  and  evaluation  of  qualitative  and  quantita¬ 
tive  maintainability  requirements.  It  also  provides  for  qualitative 
assessment  of  various  integrated  logistic  support  factors  related  to 
and  impacting  the  achievement  of  maintainability  parameters  and  item 
downtime,  e.g. ,  technical  manuals,  personnel,  tools  and  test  equipment, 
maintenance  concepts  and  provisioning. 

1.2  Application.  The  standard  is  intended  for  lise  when  verification, 
demonstration,  and  evaluation  of  maintainability  requirements  for  hard¬ 
ware  procurements  is  required.  The  verification,  demonstration,  and 
evaluation  of  achievement  of  maintainability  requirements  shall  normally 
be  conducted  in  three  (3)  phases,  as  described  in  Section  4,  and  in 
conjunction  with  verification,  demonstration,  and  evaluation  of  the 
requirements  for  total  Integrated  Logistic  Support.  Exceptions  to  the 
three  phases  shall  be  as  specified  by  the  procuring  activity, 

2.  APPLICABLE  DOCIPENTS 

The  issues  of  the  following  documents  in  effect  on  the  date  of  invitation 
for  bids  or  request  for  proposal  form  a  part  of  this  standard  to  the 
extent  specified  herein: 

STANDARDS 

MILITARY 


MIL-STD-280  Definitions  of  Item  Levels,  Item  Exchangeability, 
Models,  and  Related  Terms 

MIL-STD-470  Maintainability  Program  Requirements  (For  Systems 
and  Equipments) 

MIL-STD-721  Definition  of  Effectiveness  Terms  for  Reliability, 
Maintainability,  Human  Factors,  and  Safety 

"Copies  of  specifications,  standards,  drawings  and  publications  required 
by  suppliers  in  connection  with  specific  procurement  functions  should  be 
obtained  from  the  procuring  activity  or  as  directed  by  the  contracting 
officer." 

3.  DEFINITIONS 

fleanings  of  terms  not  defined  herein  are  in  accordance  with  MIL-STD-280 
and  MIL-STD-721. 
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3.1  Maintenance  Task.  The  maintenance  effort  necessary  for  retaining 
an  item  in,  changing  to,  or  restoring  it  to  a  specified  condition.  The 
procuring  activity  will  provide  to  the  contractor  any  terms  that  will 
be  considered  synonomous  with  the  term  task  and  will  provide  definitive 
criteria  for  determining  different  types  of  maintenance  tasks  and  the 
timing  of  the  tasks  during  verification/deraonstration/evaluation. 

3.2  Maintainability  Model.  A  quantifiable  representation  of  a  test  or 
process  the  purpose  of  which  is  to  analyze  results  to  determine  specific 
relationships  of  a  set  of  quantifiable  maintainability  parameters. 

3.3  Verification.  The  contractor  effort,  monitored  by  the  procuring 
activity,  from  date  of  award  of  the  contract,  progressing  concurrently 
through  hardware  development  from  components  to  the  configuration 
item  (Cl);  to  determine  the  accuracy  of  and  update  the  analytical 
(predicted)  data  obtained  from  the  maintainability  engineering  analysis; 
to  identify  maintainability  design  deficiencies;  and  to  gain  progressive 
assurance  that  the  maintainability  of  the  item  can  be  achieved  and 
demonstrated  in  subsequent  phases. 

3.4  Demonstration.  The  jo.,jt  contractor  and  procuring  activity  effort 
to  determine  whether  spec.f*  .  maintainability  contractual  requirements 
have  been  achieved. 

3.5  Evaluation.  The  procuring  activity  effort  to  determine,  at  all 
levels  of  maintenance,  the  impact  of  the  operational,  maintenance  and 
support  environment  on  the  maintainability  parameters  of  the  item  and  to 
demonstrate  depot  level  maintenance  tasks. 

3.6  Development  Test  and  Evaluation  (DT§E) .  Test  and  evaluation  which 
focuses  on  the  technological  and  engineering  aspects  of  the  system, 
subsystem,  or  equipment  items. 

3.7  Operational  Test  and  Evaluation  (OTf^E) .  Test  and  evaluation  which 
focuses  on  the  development  of  optimum  tactics,  techniques,  procedures,  and 
concepts  for  systems  and  equipment,  evaluation  of  reliability,  maintain¬ 
ability  and  operational  effectiveness,  and  suitability  of  systems  and  equip 
ment  under  realistic  operational  conditions. 

3.8  Maintenance  Concept.  A  description  of  the  planned  general  scheme  for 
maintenance  and  support  of  an  item  in  the  operational  environment. 

3.9  Maintenance  Environment.  The  climatic,  geographical,  physical  and 
operational  conditions  (e.g. ,  combat,  mobil,  continental)  under  which  an 
item  will  be  maintained. 
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4.  REQUIREMENTS 

4.1  General.  Maintainability  C^)  verification,  demonstration,  and 
evaluation  shall  be  performed  in  accordance  with  the  M  test  plan 
(see  4.2)  prepared  by  the  contractor  and  approved  by  the  procuring 
activity.  The  M  test  plan  shall  fom  a  part  of  the  integrated  sujiport 
plan  when  an  integrated  support  plan  is  required.  The  M  test  plan  shall 
be  prepared  and  submitted  as  part  of  the  contractor's  proposal,  and 
progressively  updated  as  design,  development,  and  fabrication  proceed. 

It  shall  be  available  for  in  process  reviev;  by  the  procuring  activity. 

Those  portions  of  the  total  ^  test  plan  applicable  to  specific  phases 
(verification,  demonstration,  evaluation)  shall  be  submitted  to  the 
procuring  activity  for  approval  prior  to  its  implementation  and  no  later 
than  the  date  specified  by  the  contract.  The  ^  test  plan  shall  be  totally 
responsive  to  the  qualitative  and  quantitative  requirements  and  supplemental 
information  contained  in  the  procurement  documents  and  the  ^  program  plan 
required  by  MflL-STD-470,  "Maintainability  Program  Requirements."  The 
supplemental  information  shall  include,  but  not  be  limited  to,  maintenance 
concept,  maintenance  environment,  skill  levels  of  personnel,  level(s)  of 
maintenance  to  be  demonstrated,  and  modes  of  operation  for  test,  including 
configuration  and  missions.  Coordination  of  the  M  verification,  demonstra¬ 
tion,  and  evaluation  with  other  required  demonstrations  shall  be 
accomplished  whenever  possible  to  avoid  unnecessary  duplication  of  effort. 
The  environment  and  procedures  shall  represent,  as  closely  as  practical, 
that  which  can  be  expected  in  the  intended  operational  use  of  the  item. 

The  plan,  when  applied  to  the  system  level,  shall  embody  the  three  (3) 
phases:  verification  (Phase  I);  demonstration  (Phase  II);  and  evaluation 
(Phase  III).  V/hen  the  plan  is  applied  to  less  than  system  level,  the 
procuring  activity  shall  specify  the  applicable  phases.  Figure  1  depicts 
a  general  tine-phase  relationship  of  the  three  (3)  phases.  It  should  be 
recognized  that  Figure  1  depicts  a  general  time-phasing  only,  which  nay 
differ  for  individual  procurements.  "Hie  procuring  activity  will  provide 
guidance  to  the  contractor  as  to  the  relationship  between  system  life 
cycle  phases  and  the  verification/demonstration/evaluation  phases.  Of 
particular  importance  to  the  accomplishment  of  the  procedures  contained  in 
this  standard  is  the  detailed  information  contained  in  the  contractor's 
maintainability  analysis  as  defined  in  'lIL-STD-470.  This  analysis  must 
contain  a  comprehensive  description  of  the  predicted  maintenance  tasks. 

For  example,  the  maintainability  analysis  shall  contain  the  following: 

a.  Failure  mode  or  symptom  and  "how  malfunction  code,"  which  would 
initiate  the  corrective  maintenance  task. 

b.  Frequency  of  occurrence  of  each  failure  mode  and  symptom  of 
every  maintenance  task. 
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c.  Appropriate  "action  taken  codes"  and  "work  unit  codes"  for 
each  maintenance  task. 

d.  Predicted  times  for  each  element  of  maintenance  time  as 
defined  in  MIL-STD-721. 

e.  Skill  levels  and  number  of  people  required  for  each  maintenance 

task. 

f.  Support  equipment  and  tools  required  for  each  maintenance  task. 

g.  Technical  order  interface  for  each  maintenance  task, 

h.  Identification  of  preventive  maintenance  tasks. 

i.  Identification  of  those  maintenance  tasks  which  are  not  normally 

or  under  any  circumstances  will  not  be  permitted  to  be  performed  concurrently 
with  otlier  maintenance  tasks.  It  is  assumed  that  all  other  maintenance 
tasks  can  be  performed  unrestricted  by  the  performance  of  on-going 
maintenance. 

4.1.1  Phase  I .  During  Phase  I,  the  contractor  shall  conduct  an 
incremental  verification  effort,  commencing  with  initial  design  and 
continuing  through  hardware  development  from  components  to  the  configura¬ 
tion  item.  The  basic  objectives  of  this  phase  arc; 

4. 1.1.1  To  verify  and  update  the  contractor's  maintainability  model, 

4. 1.1. 2  To  insure  economical  correction  of  design  deficiencies  and  to 
provide  assurance  that  maintainability  requirements  will  be  achieved 
and  demonstrated,  by  performing  early  in  the  design  process,  M  verifica¬ 
tions  such  as  limited  lov/  confidence  maintainability  tests,  time-motion 
measurements  or  such  other  tests  as  may  be  proposed  by  the  contractor, 
subject  to  approval  by  the  procuring  activity, 

4. 1.1. 3  To  provide  progressive  assurance  that  the  maintainability 
requirements  can  be  achieved  and  demonstrated  and  that  elements  of  the 
integrated  support  plan  directly  related  to  M  are  valid. 

Maximum  use  shall  be  made  of  data  resulting  from  maintenance  performed 
in  conjunction  with  such  tests  as  development,  prototype,  mock-up, 
qualification,  and  reliability  tests.  VJhen  the  procurement  documents 
specify  that  the  maintainability  demonstration  shall  be  part  of  Phase  I, 
the  M  demonstration  and  requirements  of  Phase  II  (see  4.1.2)  shall  apply. 
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4.1,2  Phase  II.  The  objective  of  this  phase  is  to  determine  during 
Development,  Test  and  Evaluation  (D,  T5E)  whether  all  specified  ^ 
contractual  requirements,  except  as  noted  under  Phase  III  have  been 
achieved.  During  this  phase,  the  procuring  activity  will  manage  and 
conduct  a  maintainability  demonstration  as  part  of  the  total  system 
demonstration.  For  those  procurements  which  do  not  require  a  total 
system  demonstration,  the  maintainability  demonstration  to  be  conducted 
during  Phase  II  shall  be  an  extension  of  Phase  I.  To  assure  acceptabil¬ 
ity  of  recorded  data  and  resultant  analysis,  the  contractor  shall 
participate  to  the  extent  provided  in  4.4  of  this  standard.  The 
following  requirements  apply  to  all  maintainability  demonstrations. 
Additional  requirements  or  changes  may  be  imposed  on  individual 
procurements . 

4. 1.2.1  The  maintainability  demonstration  shall  be  conducted  in  an 
environment  which  simulates,  as  closely  as  practicable,  the  operational 
and  maintenance  environment  planned  for  the  item.  This  environment 
shall  be  representative  of  the  working  conditions,  tools,  support 
equipment,  spares,  facilities,  and  technical  publications  that  would  be 
required  during  operational  service  use  at  the  maintenance  level  defined 
in  the  approved  maintenance  plan. 

4. 1.2. 2  Government  personnel  assigned  to  the  test  organization  shall 
operate  and  maintain  the  demonstration  items  (see  4.2.3  and  4.4.1).  iVhen 
demonstration  is  conducted  as  an  extension  of  Phase  I,  the  procuring 
activity  shall  specify  the  personnel  (Government  or  contractor)  who  will 
operate  and  maintain  the  items. 

4. 1.2. 3  In  conjunction  with  the  maintainability  demonstration,  the 
approved  integrated  support  plan,  when  required,  and  established  by  the 
contractor,  scaled  to  the  nmber  of  test  items  employed  in  the  demonstra¬ 
tion,  shall  be  implemented  by  the  test  team  to  identify  the  logistic 
support  provided  during  Phase  II. 

4. 1.2.4  All  maintenance  data,  including  depot  level,  shall  be  recorded 
and  reported  to  the  test  team  as  specified  by  the  procuring  activity. 

4. 1.2. 5  Unless  approved  otherwise  by  the  procuring  activity,  the 
configuration  of  the  items  of  the  system  selected  for  ^  demonstration 
shall  be  documented  and  certified  by  a  physical  confi^ration  audit  (PCA) . 

4. 1.2.6  Unless  approved  otherwise  by  the  procuring  activity,  all  support 
equipment  used  during  the  demonstration  shall  be  certified  by  PCA. 

4. 1.2. 7  Maintenance  tasks  which  may  require  fault  simulation  (see 
4.3. 1.2  and  4.3. 1.3)  shall  require  that  the  item  be  checked  for  normal 
operation  prior  to  failure  simulation  and  after  completion  of  the 


6 


'aL-STn-471A 
27  March  1973 

specified  maintenance  task.  Whan  a  failure  is  simulated,  it  will  be 
the  responsibility  of  the  test  team  to  select  the  maintenance  task, 
the  failure  to  be  simulated,  and  the  failure  mode;  and  to  verify  that 
the  degree  of  failure  is  representative  of  the  maintenance  task  to  be 
demonstrated.  The  work  area  in  which  parts  degradation  or  failure  has 
been  simulated  shall  contain  no  obvious  evidence  other  than  that 
normally  resulting  from  the  simulated  mode  of  failure.  The  appearance 
of  defective  parts  that  are  substituted  for  serviceable  parts  shall  be 
that  of  a  normally  failed  part.  The  technician  shall  not  witness  any 
fault  insertion.  Simulation  of  failures  by  introduction  of  faulty 
parts  will  not  be  used  when  the  normal  procedures  could  result  in 
extensive  damage  to  the  equipment  or  item  being  tested.  Each  defective 
part  is  to  be  installed  in  the  equipment  in  the  same  manner  as  the 
original  part. 

4. 1.2.8  For  maintenance  tasks,  whose  faults  have  been  simulated,  the 
presence  of  necessary  spares,  tools,  test  and  support  equipment,  or 
technical  publications  shall  not  assist  in  fault  isolation  by 
prematurely  identifying  the  work  to  be  done.  Such  items  shall  be 
covered  or  otherwise  kept  out  of  sight  from  the  technician.  However, 
simulated  discrepancy  data  shall  be  made  available,  if  applicable. 

4. 1.2.9  Maintenance  personnel  performing  maintenance  tasks  for  the 
demonstration  shall  be  military  or  civil  service  personnel,  with  the  excep¬ 
tion  that  contractor  personnel  will  perform  those  tasks  specified  to  be 
performed  by  contractual  personnel  during  the  operational  service  use 

(see  4. 1.2.2).  Technicians  shall  have  received  the  training  and  be  of 
the  equivalent  skill  level  as  specified  in  the  standard  personnel  resource 
documentation  for  the  specified  level  of  maintenance.  Exception  to  the 
training  and  skill  level  requirements  may  be  made  for  specified  tasks 
which  will  be  performed  by  contractor  personnel  during  operational 
service  use. 

4.1.2.10  Each  maintenance  task  will  be  documented  by  personnel 
designated  by  the  test  team.  The  total  time  measured  for  a  technician 
to  perform  each  maintenance  task  shall  be  recorded  and  will  include  the 
time  to  perform  each  element  of  maintenance  time  defined  in  MIL-STD-721. 
Each  element  will  be  documented  separately.  The  total  delay  time  for  each 
maintenance  task  shall  be  documented.  The  test  plan  and  procedures  shall 
include  delay  time  rules. 

4.1.2.11  The  time  required  to  obtain  support  items  (appropriate  test  and 
support  equipment,  tools,  spare  parts,  technical  publications,  etc.)  from 
the  defined  work  center  area  shall  be  recorded.  This  time  shall  not, 
however,  be  chargeable  as  maintenance  task  time  for  the  item  being 
demonstrated  unless  this  time  is  controlled  or  influenced  by  the  design  of 
the  item  being  demonstrated. 
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4.1.2.12  Items  to  be  furnished  by  the  contractor  shall  be  provided 
in  the  type,  quality,  and  quantity  required  for  planned  operation 
requirements  scaled  to  the  demonstration  and  evaluation  requirements, 
prior  to  the  start  of  the  phase  being  performed.  Items  to  be  furnished 
by  the  procuring  activity  shall  be  identified  and  requested  by  the 
contractor  in  time  to  be  available  prior  to  the  start  of  the  phase 
being  performed. 

4.1.3  Phase  III.  The  objective  of  this  phase  is  to  (1)  evaluate  the 
impact  of  the  actual  operational,  maintenance,  and  support  environment 
on  the  maintainability  parameters  of  the  system,  (2)  to  evaluate  the 
correction  of  deficiencies  exhibited  during  Phase  II,  and  (3)  to 
demonstrate  depot  level  maintenance  tasks  when  applicable.  A  maintain¬ 
ability  evaluation  will  be  managed  and  conducted,  by  the  procuring 
activity,  during  Operational,  Test  and  Evaluation  as  part  of  the  total 
system  evaluation.  To  assure  acceptability  of  recorded  data  and 
resultant  analysis,  the  contractor  shall  participate  in  Phase  III  to  the 
extent  described  in  4.4  of  this  standard  or  as  otherwise  provided.  The 
same  conditions  outlined  for  Phase  II  (see  4.1.2)  shall  apply,  except 
for  the  following; 

4.1.3. 1  All  evaluation  items  shall  be  production  or  production  equivalent 
models. 

4. 1.3. 2  TTie  evaluation  shall  be  conducted  in  the  actual  operational  and 
maintenance  environment  unless  otherwise  directed  by  the  procuring 
activity. 

4. 1.3. 3  All  maintenance  tasks  will  be  accomplished  by  military  or  civil 
service  personnel  with  the  exception  that  contractor  personnel  will 
perform  those  tasks  specified  to  be  performed  by  contractual  personnel 
during  operational  service  use. 

4. 1.3. 4  Depot  level  maintenance  tasks  shall  be  d«nonstrated  and  the 
data  collected  applied  to  the  maintainability  demonstration  and 
evaluation. 

4. 1.3.5  Maintenance  tasks  to  be  evaluated  shall  be  those  resulting 
directly  from  and  incidental  to  actual  operation  and  maintenance.  These 
tasks  shall  be  supplemented  by  fault  simulation  only  to  evaluate  specific 
tasks  or  special  tasks  (see  4. 3. 1.3)  that  do  not  occur  by  chance  during 
the  evaluation  phase. 

4.2  Maintainability  Verification/Demonstration/Evaluation  Plan.  The  plan, 
prepared  by  the  contractor  in  accordance  with  the  Contract  Data  Require¬ 
ments  List  (CDRL) ,  shall  include  the  following  sections,  as  a  minimum. 
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identified  with  eadi  of  the  three  (3)  phases,  unless  instructions  to 
the  contrary  are  provided  in  the  specific  procurement.  Certain 
sections  cover  material  subject  to  other,  more  specific,  contractual 
requirements  and  nay  be  included  in  the  plan  as  they  are  prepared  in 
response  thereto.  They  are  included  to  insure  adequate  attention  and 
continui ty. 

4.2.1  Background  Information.  A  description  of: 

4. 2. 1.1  Quantitative  and  qualitative  maintainability  requirements; 

4. 2. 1.2  Maintenance  concept; 

4. 2. 1.3  Maintenance  environment; 

4. 2. 1.4  Level(s)  of  maintenance; 

4. 2. 1.5  Sites; 

4. 2. 1.6  Facilities*  requirements; 

4. 2. 1.7  Participating  agencies; 

4. 2. 1.8  ModeCs)  of  operation  of  the  items,  including  configuration 
and  mission  requirements; 

4. 2. 1.9  Items  subject  to  verification,  demonstration  and  evaluation;  and 

4.2.1.10  Contractual  data  required  for  completion  of  the  verification/ 
demonstration/ evaluati on. 

4.2.2  Item  Interfaces.  A  description  of  the  adequacy  or  inadequacies  of 
the  item  siqpport  elements  and  an  estimate  of  their  effect  on  the  item 
maintainability.  These  elements  would  include  the  following: 

4. 2. 2.1  Maintenance  planning; 

4. 2. 2. 2  Support  and  test  equipment; 

4. 2. 2. 3  Supply  support; 

4. 2. 2. 4  Transportation,  handling  and  storage; 

4. 2. 2.5  Technical  data; 

4. 2. 2.6  Facilities;  and 

4. 2. 2. 7  Personnel  and  training. 
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4.2.3  Test  Team.  A  description  of: 

4. 2. 3.1  Organization; 

4. 2. 3. 2  Degree  of  contractor  and  procuring  activity  participation, 
including  managerial,  technical,  maintenance,  and  operation  personnel; 

4. 2. 3. 3  Assignment  of  specific  responsibilities;  and 

4. 2. 3. 4  Qualifications,  quantity,  sources,  training,  and  indoctrination 
requirements  for  the  test  team  personnel. 

4.2.4  Support  Material.  A  description  of: 

4 . 2 . 4 . 1  Support  equipment ; 

4. 2. 4. 2  Tools  and  test  equipment; 

4. 2. 4. 3  Technical  manuals; 

4. 2. 4.4  Spares  and  consumables; 

4. 2.4.5  Safety  equipment;  and 

4. 2. 4.6  Calibration  equipment. 

4.2.5  Preparation  Stage.  A  description  of  and  schedule  for: 

4. 2.5.1  Organization  and  assembly  of  the  test  team; 

4. 2. 5. 2  Training  of  personnel; 

4. 2. 5. 3  Preparation  of  facilities;  and 

4. 2. 5. 4  Availability,  assembly,  checkout,  and  preliminary  validation  of 
support  material, 

4.2.6  Verification/Demonstration/Evaluation  Stage.  A  description  of: 

4. 2.6.1  Test  objectives; 

4. 2.6. 2  Schedule  of  tests; 

4. 2.6.3  Procedure  for  selection  of  maintenance  tasks  when  faults  are 
simulated  (see  4. 3. 1.2); 
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4. 2. 6. 4  Identification  of  special  maintenance  tasks  (see  4. 3. 1.3); 

4. 2.6.5  Test  method,  including  accept/reject  decision  criteria,  risks,  etc. 

4. 2.6.6  Data  acquisition  method; 

4. 2. 6. 7  Data  analysis  methods  and  procedures; 

4. 2.6.8  Specific  data  elements; 

4. 2. 6. 9  Units  of  measurement; 

4.2.6.10  Type  and  schedule  of  reports; 

4.2.6.11  Schedule  of  maintenance  task  accomplishment  such  as  time  change 
compliance  tasks,  inspection,  lubrication,  and  turn  around  tasks;  and 

4.2.6.12  The  maintenance  tasks,  other  than  those  listed  in  4.2.6.11,  to 
be  verified,  demonstrated,  and  evaluated.  These  tasks  may  be  prepared 
and  submitted  in  a  referenced  document. 

4,2.7  Retest  Stage.  A  provisional  schedule  for  special  or  repeat  tests 
to  investigate  deficiencies  or  trouble  areas.  Deficiencies  shall  be 
corrected  in  any  item  which  has  failed  to  meet  the  acceptance  criteria. 

The  corrected  portions  of  the  item  and  any  other  portions  of  the  item 
affected  by  the  correction  shall  be  retested  during  this  stage.  The 
maintenance  tasks  to  be  demonstrated  shall  be  as  designated  by  the 
procuring  activity. 

4.3  Test  Procedures.  In  designing  the  maintainability  test  procedures, 
both  qualitative  and  quantitative  requirements  shall  be  verified, 
demonstrated,  and  evaluated.  Unless  instructiore  to  the  contrary  are 
provided  in  the  specific  procurement  contractual  documentation, 
qualitative  maintainability  requirements  will  be  verified,  demonstrated, 
and  evaluated  using  contractor  prepared  checklists.  These  checklists, 
to  be  approved  by  the  procuring  activity,  will  permit  observation, 
analysis,  and  identification  of  maintainability  characteristics 
incorporated  or  omitted.  Quantitative  requirements  shall  be  verified, 
demonstrated,  and  evaluated  by  actual  demonstration  of  maintenance  tasks, 

4.3.1  Maintenance  Task  Generation.  All  maintenance  tasks  shall  be 
performed  at  the  maintenance  level  approved  by  the  procuring  activity  and 
in  accordance  with  the  approved  maintenance  plan.  Maintenance  tasks,  both 
corrective  and  preventive,  shall  be  generated  by  the  following  methods  as 
identified  in  the  final  approved  maintainability  verification,  demonstra¬ 
tion,  and  evaluation  plan. 
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4. 3. 1.1  Actual  operation  of  the  item  in  the  specified  test,  operational, 
and  maintenance  environment.  Tliis  method  is  preferred,  provided  that 
assurance  can  be  given  that  sufficient  number  of  maintenance  tasks  will 
occur  during  the  test  period  to  satisfy  the  minimum  sample  requirements 
for  the  test  method  employed  (see  Appendix  B) . 

4.3. 1.2  Fault  simulation  by  introduction  of  faulty  parts,  deliberate 
misalignment,  open  leads,  shorted  parts,  etc.  A  maintenance  task 
sampling  plan  shall  be  prepared  by  the  contractor  in  accordance  with 
the  procedure  described  in  Appendix  A  or  as  directed  by  the  procuring 
activity  for  approval  by  the  latter.  The  actual  task  selection,  by  the 
test  team,  shall  not  be  accomplished  until  immediately  prior  to  the 
demonstrati on. 

4.3. 1.3  "Special"  maintenance  tasks  which  require  unique  skills, 
equipment,  test  methods,  etc.,  will  be  selected  by  the  procuring 
activity.  The  method  of  demonstrating  these  tasks  will  be  specified 
by  the  procuring  activity. 

4.3.2  Turnaround  Tasks.  Tasks  comprising  tumaroiond  shall  be 
demonstrated.  These  tasks  shall  be  determined  from  the  planned 
operational  use  of  the  item. 

4.3.3  Test  Method.  Statistical  test  methods  and  criteria  for  deciding 
whether  specified  maintainability  requirements  have  been  met  are  described 
in  Appendix  B.  Guidance  on  selection  and  application  of  the  test  methods 
is  included  with  each.  Selection  of  the  test  method  shall  be  from 
Appendix  B,  subject  to  procuring  activity  approval  or  as  otherwise 
specified. 

4.4  Administration.  The  following  shall  apply  in  the  administration  of 
the  verification,  demonstration  and  evaluation  of  the  maintainability  of 
the  item. 

4.4.1  Test  Team  Responsibility.  The  procuring  activity/ con tractor 
verification,  demonstration,  and  evaluation  team(s)  for  each  of  the 
three  (3)  applicable  phases  shall  be  empowered  to  make  decisions  for 
their  respective  organizations.  Each  member  of  the  team  may  have 
advisors  from  his  organization  who  are  knowledgeable  in  the  various 
aspects  of  the  demonstration  and  the  requirements  of  the  verification/ 
demonstration/evaluation  plan.  The  responsibilities  of  the  team  are  in 
accordance  with  the  contractors  approved  maintainability  verification/ 
demonstration/evaluation  plan  and  shall  include,  but  are  not  limited 
to  the  following: 
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4.4. 1.1  To  maintain  surveillance  over  maintenance  and  inspection 
operations.  Any  apparent  discrepancies  in  maintenance  task  accomplish¬ 
ment  and  documentation  observed  by  any  member  of  the  team  will  be 
brought  to  the  attention  of  the  remaining  test  team  members  within 

one  working  day  of  the  occurrence  for  appropriate  action. 

4.4. 1.2  To  evaluate  and  validate  maintenance  and  operational  data  to 
determine  applicable  manhours,  flying  hours,  operating  time,  maintenance 
time,  downtime,  item  status,  etc, 

4. 4. 1.3  To  assure  that  the  demonstration  item  selected  has  been 
adequately  prepared  in  accordance  with  applicable  technical  manuals 
and  that  no  maintenance  has  been  deferred  that  will  compromise  the 
successful  completion  of  the  next  scheduled  operation  or  mission 
prior  to  being  placed  in  an  operational  ready  status. 

4. 4. 1.4  To  decide  if  resulting  failures,  maintenance  time,  elapsed 
downtime,  maintenance  manhours,  etc.,  should  be  chargeable  in  cases 
where  operator  or  maintenance  crew  errors  have  been  committed. 

4.4. 1.5  To  rule  on  questions  of  whether  or  not  the  verification, 
demonstration,  and  evaluation  plan  has  been  adhered  to. 

4.4. 1.6  To  rule  on  controversial  points  which  may  arise  that  are  not 
specifically  covered  by  applicable  specifications  or  other  pertinent 
documentation.  To  determine  those  matters  which  require  contractual 
interpretation  or  resolution  by  the  appropriate  government  and 
contractor  organizations.  For  these  matters,  the  test  team  majority 
and  minority  statements  shall  be  submitted  to  the  procuring  activity 
contracting  officer  for  resolution. 

4.4. 1.7  To  prepare  and  submit  demonstration  status  reports  to  the 
procuring  activity  and  the  contractor. 

4.4.1. 8  To  analyze  data  and  determine  the  extent  of  achievement  of 
specified  maintainability  requirements. 

4. 4. 1.9  To  prepare  and  submit  final  results  of  each  of  the  phases  to 

the  procuring  activity  and  the  contractor  within  the  time  period  indicated 
in  the  approved  test  plan, 

4.4.1.10  To  assure  that  the  following  conditions  have  been  fulfilled 
prior  to  the  start  of  Phase  II  and  Phase  III  and  that  a  letter  has  been 
sent  to  the  procuring  activity  which  so  attests. 

4,4.1.10,1  Each  test  item  complies  with  the  established  configuration 
or  that  all  deviations  reported  have  been  accepted  by  the  procuring 
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activity.  It  shall  be  the  responsibility  of  the  contractor  to 
report  all  deviations  from  the  approved  configuration. 

4.4.1.10.2  All  required  technical  manuals  have  been  updated  as 
necessary. 

4.4.1.10.3  The  support  resources  are  available  in  the  type  and 
quantity  specified  in  the  verification*  demonstration,  and 
evaluation  plan. 

4.4.1.10.4  All  operator  or  maintenance  crew  personnel  are  properly 
trained  and  meet  established  skill  level  requirements. 

4.4.1.10.5  All  records  of  approved  changes  in  personnel  requirements, 
operating  and  maintenance  manuals,  data  handling  procedures,  and 
analysis  techniques  have  been  incorporated  in  the  final  revision  of 
the  verification,  demonstration,  and  evaluation  plan. 

4.4.2  Test  Director.  An  individual,  designated  by  the  procuring 
activity,  as  test  director,  shall  decide  in  all  cases  of  deadlock 
between  the  members  of  the  team  (subject  to  contract  negotiations  where 
contractual  obligations  are  in  question). 

4.4.3  Instrumentation  Failures.  Any  failures  of  test  instrumentation 
used  to  instrument  the  demonstration  it«n  for  test  purposes  or  failures 
induced  by  such  test  instrumentation  installation  or  operation,  and  all 
associated  maintenance,  shall  not  be  chargeable. 

4.4.4  Maintenance  Due  To  Secondary  Failures.  If  any  secondary  failures 
result  frm  a  chargeable  primary  failure,  tKe  total  resultant  maintenance 
time  to  restore  the  items  shall  be  chargeable  as  a  single  maintenance, 
task,  except  when  the  secondary  failure  results  from  the  method  used  to 
simulate  a  fault  rather  than  from  the  fault  itself.  If  the  reason  for 
the  secondary  failure  is  removed  (corrected),  the  time  charge  for  the 
secondary  failure  shall  be  deleted. 

4.4.5  Inadequate  Technical  Manuals  Or  Support  Equipment.  If,  in  the 
accomplishment  of  a  maintenance  task,  a  technician  finds  the  applicable 
technical  manuals  or  svipport  equipment  to  be  inadequate,  these  instances 
shall  be  brought  to  the  attention  of  the  test  team  and,  if  the  inadequacy 
is  verified,  this  portion  of  the  demonstration  shall  be  terminated 

and  times  measured  shall  not  be  chargeable.  Action  shall  be  taken  to 
correct  the  inadequacies  of  the  technical  manuals  or  support  equipment, 
after  which  the  same  maintenance  task  shall  be  repeated. 
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4.4.6  Cautions,  If  an  item  is  damaged  or  maintenance  errors  induced 
by  item  design  complexity,  by  poor  design  practice,  or  by  following 
improper  procedures  that  allow  improper  maintenance  (e.g.,  inter¬ 
changeability  of  connectors)  without  proper  caution  in  the  technical 
manuals,  the  failure  and  resultant  maintenance  times  shall  be 
chargeable.  Action  shall  be  taken  to  correct  the  improper  procedures 
or  deficiencies  and  the  corrective  action  verified.  When  this  action 
is  completed,  the  maintenance  time  saved  shall  be  deleted. 

4.4.7  Personnel  Number  and  Skill.  Each  task  shall  be  performed  by 
the  prescribed  number  of  personnel  with  the  prescribed  specialty  codes 
and  skills.  If  personnel  are  required  on  an  intermittent  or  sequenced 
basis,  the  manhours  assessed  against  the  maintenance  task  will  include 
the  required  standby  time  only  if  the  standby  time  is  of  a  type  or 
duration  which  prevents  standby  personnel  from  performing  other 
productive  tasks. 

4.4.8  Cannibalization.  The  maintenance  associated  with  the  removal  or 
reinstallation  of  the  item  or  support  equipment  assemblies  and/or 
components  for  cannibalization  purposes  shall  not  be  chargeable  unless 
the  deficiency  can  be  directly  related  to  lack  of  contractor  recommenda¬ 
tions  for  proper  level  of  support  spares  or  expendables.  If  the 
contractor  takes  action  to  correct  the  deficiency,  the  time  charged 
shall  be  deleted, 

4.4.9  Availability.  An  item  shall  be  considered  in  an  operationally 
available  or  operationally  ready  status  (for  aircraft)  if  it  is  capable 
of  performing  in  accordance  with  the  item's  specification  or  capable 

of  performing  the  next  scheduled  assigned  mission. 

4.4.10  Maintenance  Inspection.  The  look  portion  of  any  inspection 
such  as  pre-flight,  post-flight,  or  phase  of  a  phased  inspection  shall 
be  considered  a  separate  preventive  maintenance  task.  Each  fix  of  the 
fix  portion  of  an  inspection  shall  be  considered  a  separate  corrective 
maintenance  task. 

4,5  GFE/GFAE  Items.  For  Government  Furnished  Equipment  (GFE)  and 
Government  Furnished  Aeronautical  Equipment  (GFAE)  items,  the  contractor 
is  responsible  for  determining  the  GFE/GFAE  maintainability  characteris¬ 
tics  and  values  required  for  his  Configuration  Item  (Cl) ,  and  for 
assuring  that  the  GFE/GFAE  maintainability  characteristics  and  values 
are  not  degraded  unless  compensated  for  by  the  demonstrated  characteristics 
and  values  for  other  Contractor  Furnished  Equipment  (CFE)  or  GFE/GFAE, 

The  government  will  furnish  data  on  known  or  estimated  values  of  GFE/GFAE 
reliability  and  maintainability  which  shall  be  used,  as  applicable,  in  the 
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contractor's  judgment.  The  contractor  is  responsible  for  estimating 
and  demonstrating  the  maintainability  requirements  of  the  entire  Cl. 

4.6  Data  Collection.  The  data  collection  system  used  in  Phase  I  and 
data  elements  collected  shall  meet  the  needs  of  the  objectives  of 
Phase  I,  In  addition,  the  data  system  and  data  elements  shall  be 
compatible  with  the  data  system  used  and  data  elements  collected  in 
Phases  II  and  III,  During  Phase  II  and  Phase  HI,  the  test  team  shall 
establish  and  operate  a  data  center.  All  data  recorded  by  the  test 
team  shall  be  made  available  to  the  contractor  through  the  data  center. 

The  test  team  shall  utilize  the  data  system  specified  by  the  procuring 
activity,  to  record  all  mission  debriefing,  failure  and  maintenance 
data.  The  contractor  shall  describe  maintenance  tasks  in  a  manner 
which  will  allow  proper  identification  within  the  services  maintenance 
data  collection  system  that  a  particular  task  has  occurred.  For 
example,  when  using  the  System  Effectiveness  Data  System  (SEDS) ,  the 
maintenance  task  description  must  contain  a  Work  Unit  Code,  How 
Malfunctioned,  and  Action  Taken  Code  which  uniquely  identify  that  task. 
Supplementary  data  collection  may  be  incorporated  if  approved  by  the 
procuring  activity.  For  those  items  which  the  contractor  has  depot 
level  repair  responsibilities,  he  shall  be  responsible  for  preparation, 
accuracy,  and  feedback  of  the  depot  level  verification,  demonstration, 
evaluation  maintenance  data  for  all  depot  repairables  generated.  All 
depot  level  data  elements  collected  shall  be  compatible  with  the  data 
elements  collected  and  recorded  at  the  organizational  and  intermediate 
maintenance  levels.  All  direct  maintenance  downtime  or  manhours,  as 
applicable,  which  is  not  specifically  determined  to  be  nonchargeable  shall 
be  included  in  the  demonstration  data  and  in  the  calculated  quantitative 
value  which  determines  compliance  or  nonconpliance.  Maintenance  which 
might  not  be  chargeable  could  result  from  such  causes  as; 

4.6.1  Maintenance  and  operational  errors  not  chargeable  to  technical 
manuals,  contractor  furnished  training  or  faulty  design. 

4.6.2  Miscellaneous  tasks  such  as  keeping  of  records,  taxiing  and  towing 
of  aircraft  to  or  from  an  area  other  than  the  assigned  work  center  area, 

4.6.3  Repair  of  accident  damage. 

4.6.4  Docimiented  delay  downtime  (supply  or  administrative)  which  is 
clearly  outside  the  responsibility  of  the  contractor, 

4.6.5  Modification  tasks. 

4.6.6  Maintenance  of  test  instrumentation  exclusive  of  normal 
configuration. 
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4.6.7  Maintenance  tine  accountable  to  test  instrumentation  installations 
(other  than  normal  configuration)  accrued  during  maintenance  task 
performance. 

4.7  Maintainability  Parameter  Calculations.  All  data  acceptable  to  the 
team  and  generated  by  the  demonstration  shall  be  used  in  calculating  the 
M  parameters.  The  following  are  typical  maintainability  parameters  which 
may  be  stated  in  the  specification:  Mean-Time-To-Repair  (MTTR) ,  manhour 
rate,  critical  maintenance  time  or  manhours,  critical  percentile,  and 
chargeable  maintenance  downtime  (a  parameter  for  demonstration  of  avail¬ 
ability).  Appendix  B  provides  methods  for  calculating  these  values  and 
the  criteria  for  determining  whether  the  requirements  have  been  achieved. 
Other  methods  of  calculation  tailored  to  a  specific  procurement  may  be 
provided/ approved  by  the  procuring  activity. 

4.8  Maintainability  Verification/Demonstration/Evaluation  Reports.  A 
final  report  shall  be  submitted  by  the  test  team,  a:^ter  each  phase,  to 
the  procuring  activity  in  accordance  with  the  schedule  incorporated  in 
the  verification/ demonstration/evaluation  plan  and  the  data  requirements 
per  Contractor  Data  Requirements  List  (DD  Form  1423).  The  procuring 
activity  may  require  interim  reports  where  additional  detail  or  extended 
test  durations  may  be  involved.  The  final  report  shall  include,  as  a 
minimum,  the  following: 

4.8.1  Summary  of  data  collected  and  location  of  data  file. 

4.8.2  Factors  which  influence  the  data. 

4.8.3  Analysis  of  the  data. 

4.8.4  Results  of  the  phase  and  certification  that  the  specified  objectives 
and  requirements  have  or  have  not  been  met, 

4.8.5  Assessment  of  the  integrated  logistic  support  factors,  such  as 
technical  manuals,  personnel,  tools  and  test  equipments,  support  equip¬ 
ment,  maintenance  concept  and  provisioning  for  their  effect  on  quantita¬ 
tive  and  qualitative  demonstrated  maintainability  parameters. 

4.8.6  Deficiencies. 

4.8.7  Recommendations: 

4. 8. 7.1  to  correct  deficiencies  and 

4. 8. 7. 2  for  suggested  improvements. 

4.8.8  Results  of  retest  (if  applicable).  To  be  submitted  as  a  supplement 
to  the  final  report. 
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5.  ORDERING  DATA 

The  selected  data  requirements  in  support  of  this  standard  shall  be 
reflected  in  the  Contractor  Data  Requirements  List  (DD  Form  1423) 
attached  to  the  Request  for  Proposal,  Invitation  for  Bid,  or  the  Contract, 
as  appropriate.  The  following  information  will  be  included  in  the 
applicable  contractual  documents: 

a.  Phases  applicable  to  the  procurement  (see  1.2). 

b.  Dates  for  submission  of  the  test  plan  and  test  procedures  for 
each  phase  (see  4.1,  4.2). 

c.  Type  of  personnel  (government  or  contractor  who  will  operate 
and  maintain  the  item  for  maintainability  demonstration)  (see  4. 1.2. 2). 

d.  Dates  for  submission  of  the  final,  interim  and  supplemental 
(if  required)  reports  for  each  phase  (see  4.8). 

e.  Data  collection  system  (4.6). 

f.  Specification  Requirements  and  Test  ’lethod  (see  Appendix  B; 

B.10.2  for  major  characteristics  for  the  test  method  specified). 


CUSTODIANS 

Army  -  EL 
Navy  -  AS 
Air  Force  -  17 

REVIEW  ACTIVITIES 


PREPARING  ACTIVITY 

Air  Force  -  17 
Project  MISC-0855 


Army  -  EL,  MI,  SC,  TE 
Navy  -  EC 

Air  Force  -  10,  11,  13,  15,  22,  26 

USER  ACTIVITIES 

Army  - 
Navy  - 

Air  .Fprce  -  19,  71,  80 
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APPENDIX  A 

MAINTENANCE  TASK  SAMPLING  FOR  USE  WITH  FAILURE  SIMULATION 
A.  10  Scope. 

A. 10.1  Purpose .  This  appendix  outlines  a  procedure  for  the  selection  of 
a  sample  of  corrective  maintenance  tasks  for  maintainability  demonstration 
when  the  tasks  result  from  failure  simulation. 

A. 10. 2  Application.  The  procedure  described  herein  is  applicable  only 
when  failure  simulation  is  to  be  used  to  generate  maintenance  tasks.  The 
procedvire  is  applicable  to  the  equipment  level  and  it  is  assumed  that 
system  level  maintainability  requirements  have  been  allocated  to  the 
equipment  level  for  demonstration.  The  mean  estimates  for  equipment  may 
be  employed  to  determine  achievement  of  system  maintainability  requirements. 

If  sampling  of  preventive  maintenance  tasks  or  servicing  is  permitted,  a 
procedure  and  tables  similar  to  that  illustrated  in  this  appendix  for 
corrective  maintenance  must  be  developed  for  each  type  of  task  (i.e.,  preven¬ 
tive  maintenance,  servicing). 

A. 10. 3  Sample  Stratification.  The  major  objectives  of  stratification  in 
this  standard  are  to:  (a)  allow  for  the  selection  of  maintenance  tasks  in  such 
a  manner  that  the  selection  simulates  the  failure  frequency  of  the  test  unit  in 
actual  operation,  (units  with  low  MTBF's  will  be  selected  more  frequently 
than  units  with  higher  MTBF's),  (b)  insure  that  a  proportionately  representative 
sample  of  task  types/times  are  selected.  Proportional  stratified  sampling 
may  be  used  for  selection  of  maintenance  tasks  to  be  demonstrated  using  the 
fixed  sample  size  test  methods  described  in  Appendix  B.  Sequential  test  method 
shall  employ  simple  random  sampling. 

A. 10. 4  Stratification  Procedure .  The  following  example  illustrates  the 
procedure  for  tasks  which  would  be  classified  as  corrective  maintenance. 
Preventive  maintenance  or  servicing  tasks  should  not  be  combined  with 
corrective  maintenance  tasks  for  the  purpose  of  task  stratification.  For 
system  level  demonstration  of  maintainability  requirements,  the  procedure 
should  be  applied  to  each  contract  end  item  equijjment  and  through  appropriate 
techniques,  the  achievement  of  system  maintainability  requirements  may  be 
demonstrated.  Maintenance  tasks  may  be  performed  conc\irrently  or  serially 
provided  that  provision  has  been  made  to  record  the  expended  maintenance 
time  for  each  maintenance  task.  The  requirement  to  be  demonstrated  shall 
determine  the  manner  in  which  the  data  shall  be  analyzed.  The  following, 

Table  I,  illustrates  the  application  of  this  procedure  to  a  radar  equipment 
consisting  of:  Antenna,  Receiver /Transmitter,  Frequency  Tracker,  Radar  Set 
Control,  and  Drift  Angle  Indicator: 

Supersedes  page  19  of  27  March  1973 
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a.  Col;imn  1  -  Identify  the  major  units  which  comprise  the  equipment. 

b.  Column  2  -  Subdivide  each  unit  to  the  functional  level  at  which 
maintenance  for  the  demonstration  is  to  be  performed  in  accordance  with 
the  approved  maintenance  plan.  This  level  may  be  an  assembly^  module, 
printed  circuit  card  or  piece  part. 

c.  Columns  3  &  4  -  For  each  functional  level  of  maintenance  identified 
in  Column  2,  identify  in  Column  3  the  type  of  maintenance  task  or  tasks 

to  be  performed  and  in  Column  4  the  estimated  mean  maintenance  time  for  the 
task.  The  maintenance  task  time  shall  include  the  time  to  perform  each 
element  of  maintenance  time  as  defined  in  MIL-STD-721B.  The  maintenance 
tasks  and  estimated  maintenance  time  would  be  derived  from  a  maintenance 
engineering  analysis,  a  maintainability  prediction  effort,  or  from  historical 
data.  The  same  maintenance  task,  such  as  "remove  and  replace"  of  a  module 
may  result  from  different  faults  within  the  module.  Column  3  would  identify 
the  maintenance  task  and  not  the  fault  or  failure  which  results  in  the 
occurrence  of  the  task. 

d.  Column  5  -  Determine  the  failure  rate  (F/10^  hr. )  for  each  module, 
printed  circuit  card,  etc.,  for  which  the  maintenance  task  was  identified 
in  Column  3.  The  failure  rates  used  shall  be  the  latest  available  from  the 
associated  reliability  program.  If  there  is  no  reliability  program,  the 
failure  rates  may  be  selected  or  extrapolated  from  sources  approved  by  the 
procuring  activity. 

e.  Column  6  -  Determine  the  quantity  of  items  in  each  major  unit 
associated  with  each  task  in  Column  3. 

f.  Column  7  -  Determine  the  duty  cycle  for  each  item  associated 
with  each  task  in  Colvimn  3  (e.g.,  operating  time  of  a  receiver  to  the 
operating  time  of  the  radar;  engine  operating  hours  to  aircraft  flight 
hours ) . 


g.  Column  8  -  Group  together  the  maintenance  tasks  identified  in 
Column  3  which  have  both: 

(1)  Similar  maintenance  actions.  NOTE:  A  maintenance  action 
is  an  element  of  a  maintenance  task.  Although  the  estimated  maintenahce 
time  for  different  maintenance  tasks  may  be  similar,  the  actions  may  be 
different,  that  is,  one  task  may  involve  significant  dieignostics  and 
another  involve  minimum  diagnostics  but  significant  access  time. 

(2)  Similar  estimated  maintenance  times .  The  maintenance  times 
in  each  group  shall  be  within  a  range  that  shall  not  exceed  the  anallest 
value  in  the  group  by  more  than  50  percent. 
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Task  grouping  shall  be  limited  to  within  maior  units  identified  in 
Column  1. 

h.  Column  9  -  Determine  the  total  failure  rate  for  each  task 
growing  identified  in  Column  8.  The  total  failure  rate  is  equal 

to  tiie  sum  of  the  products  of  Columns  5x6x7  for  all  tasks  within 
the  group. 

i.  Column  10  -  Determine  the  relative  frequencv  of  occurrence 
for  each  task  grotping  by  dividing  the  sum  of  the  total  failure  rate 
(sum  of  Column  9)  into  tlie  individual  total  failure  rate  for  each  group. 

j.  Column  11  -  Fixed  Sample  -  A  sample  of  maintenance  tasks 
equal  to  at  least  four  times  the  sample  size  specified  for  the 

selected  test  method  (Appendix  B)  or  as  specified  by  the  procuring  acti\n.ty, 
shall  be  allocated  among  the  task  groups  in  accordance  wdth  the  relative 
frequency  of  occurrence  of  the  task  group.  Example ;  Assume  t!ie  test  metliod 
to  be  employed  requires  that  a  sample  of  50  maintenance  tasks  be  demonstrated, 
a  sample  of  200  tasks  (4  x  50)  shall  be  allocated  among  the  task  grotips  as 
follows : 


Groip  1  -  .177  X  200  =  35  tasks; 
Group  2  -  .178  x  200  ■  36  tasks; 
Groip  3  -  .016  X  200  *  3  tasks; 


Groip  7  -  ,013  X  200  "  3  tasks, 

This  allocation  is  shovm  in  Column  11.  The  maintenance  tasks  allocated 
to  each  group  shall  be  randomly  selected  and  identified  from  the  population 
of  maintenance  tasks  applicable  to  that  group.  The  total  number  of  maintenance 
tasks  which  must  be  identified  for  the  equipment  must  be  equal  to  or  greater 
than  four  times  the  demonstration  sample  size  (i.e.,  greater  than  4  x  50  =  200 
for  this  example)  in  order  that  the  number  of  tasks  identified  wdth  each  group 
is  sufficient  such  that  the  allocation  of  tasks  to  each  groujj  (i.e.,  35  task^ 
for  Group  1;  36  tasks  for  Group  2,  etc.)  may  be  randomly  selected  from  the 
population  of  tasks  identified  as  applicable  to  that  group.  The  maintenance 
tasks  which  have  been  randomly  selected  shall  not  be  returned  to  the  sample 
pool.  When  a  task  group  consists  of  more  than  one  module  or  assembly,  etc., 
such  as  group  2  of  Table  I,  the  maintenance  tasks  assigned  to  the  group 
(Column  11,  36  tasks  for  this  example)  shall  be  allocated  "to  the  modules, 
assemblies,  etc.,  within  the  group  in  accordance  with  the  relative  frequencry 
of  occurrence  of  maintenance  for  each  module,  etc.,  ivithin  the  group.  The 
procedure  would  be  the  same  as  that  used  to  deteimine  the  relative  frequency 
of  occurrence  of  the  task  groups  (Column  10)  but  would  be  applied  to  the 
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modules,  etc.,  within  the  group.  This  is  illustrated  belcwv  with  the  allocaticm 
shoOTi  included  in  Table  I,  Column  11,  Groiqi  2. 


Group  2 

Total 

Failure  Rate 

Relative 

Freq.  of  Occ, 

Danonstration 

Population 

Allocation 

A- IF- A 

23 

.217 

7.8se8  (.217  x  36 

B-IF-B 

21 

.198 

7,4^7 

C-Amplifier 

21 

.198 

7. 1^:7 

D->-fodulator 

18 

.170 

6,0=6 

E-Power  Supply 

23 

.217 

7.8b8 

TUT 

Tjm 

3^ 

k.  Column  12  -  The  maintenance  tasks  to  be  demonstrated  (50  tasks 
for  this  example)  shall  be  allocated  among  the  task  groups  in  accordance 
with  the  relative  frequency  of  occurrence  of  maintenance  for  the  group. 

Example; 

Group  1:  .177  x  50  «  8.8Se*9  tasks; 

Grcwp  2:  .178  x  50  «*  8.9ft#^9  tasks; 

Group  3:  .016  x  50  ■  .80^1  task. 


Group  7:  .013  x  50  =  .65;*1  task 

If  a  task  group  consists  of  more  than  one  module,  asseni>ly,  etc.,  sudi  as 
grot?)  2,  T^le  I,  the  maintenance  tasks  to  be  danonstrated  from  the  grmip  (coltsm  12, 
9  tasks  for  this  exauqile)  shall  be  allocated  to  the  i!K>dules,  ass«i4)lies,  etc,, 
witMn  the  group  in  accordance  with  tlie  relative  frequency  of  occurrwice  of 
maintenance  for  each  module,  etc,,  within  group.  This  is  illustrated  below 
with  the  saiple  allocation  shwvn  included  in  Table  I,  column  12, 


Relative  Freq,  Demonstration 


Group  2 

of  Occurrence 

Sample  Size 

IF-A 

.217 

1.9555  C217  X  9  -  1.95  ) 

IF-B 

.198 

1.78t5 

An?>lifier 

.198 

1.78'C»2 

'io^lator 

.170 

1.53!?'1 

Poiver  Supply 

.217 

1.95t^ 

7  total 
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The  maintenance  task  to  be  demonstrated  shall  be 

the  maintenance  tasks  allocated  to  the  group  or  nodules,  assemblies,  etc., 
Sthir^he  group  or  modules,  assemblies,  etc.,  within  the 

The  maintenance  task  to  be  demonstrated  shall  not  be  3®  nernitted 

sample  pool  and  shall  be  demonstrated  once  only  unless  otherwise  permitt 

by  the  procuring  activity. 

1.  Column  13  -  Variable  Sample/Sequential  Test  -  hTien  variable 
sample  size,  sequential  test  methods  are  employed  a  simple  ^ 

of  the  total  population  of  maintenance  tasks  ^  random  nu  bl  ^ 

based  on  a  uniform  distribution  from  0  to  1  shall  be  used.  Using  Tab 
columns  1  through  10  determine  from  the  relative  frequency  ^ 

^column  10),  the  cumulative  range  of  frequency  of  occurrence  0 
groir  A  i^intanance  task  is  selected  from  that  group  f 
Lnge  of  frequency  of  occurrence  includes  the  number  selected  P^om  the 

random  number  table.  The  number  selected  from  the  ’^^^^°^/2nber  T^e 
shall  be  "returned"  to  the  table  before  selecting  a  second  nuraber. 
"specimen"  task  demonstrated  shall  be  returned  to  the  sample  pool. 

A  10  5  Failure  Mode  Selection.  A  failure  mode  and  effect  analysis  CFl-JEA)  , 

applied  to  the  tWctional  l'ev¥l  at  which  KoS''"'eJc  ) 

shall  be  used  to  determine  the  failure  modes  or  faults  * 

which  will  result  in  the  occurrence  of  the  maintenance  T^^k  of  interest. 

To  avoid  duplication  of  effort,  the  i -itA  ^naii  uriiJ-zc  , 

be  coordinated  with  the  reliability  program  efforts. 

frequency  of  occurrence  of  the  failure  mode  will  determine  the  fault 

to  be  simulated.  This  procedure  is  illustrated  in  Table  II. 


a. 


Column  1  -  Identify  the  maintenance  task  of  interest. 


b.  Column  2  -  Determine  the  failure  modes  which  will  result  in  the 
maintenance  task  of  interest. 

c.  Column  3  -  Determine  the  effect  of  each  failure  mode  identified 
in  column  2. 

d.  Column  4  -  Determine  the  relative  frequency  of  occurrence  of  each 
failure  mode. 

e  Column  5  -  Simple  Random  Sampling  -  Determine  the  cumulative  range  of 
frequency  of  occurrence  for  each  failure  mode.  Using  a  random  number  table 
niimSpr  is  selected  and  the  failure  mode  to  be  induced  is  that  whose 

cumulative  range  of  frequency  of  occurrence  f  ®  tiie 

Th*.  niiriher  selected  from  the  random  number  table  shall  be  returned  ^o  tne 
r*irfefo4  sefeciing  a  second  nuBber.  The  specimen  deBonstrated  shall  be 

returned  to  the  sample  pool. 
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TEST  METHODS  AND  DATA  ANALYSIS 


B.IO  Scope ■ 

B.10.1  Pxirpose  -  This  appendix  contains  test  methods  and  criteria  for 
demonstrating  the  achievement  of  specified  quantitative  maintainability 
requirements:  Some  of  the  test  methods  included  are  identical  to  test 
methods  contained  in  previous  versions  of  this  Standard.  Table  lA  indicates 
the  correspondence  of  the  test  methods  included  in  this  Standard  to  those 
included  in  past  versions. 

B.IO. 2  Application  -  The  following  matrix  (Fig.  B-l)  summarizes  the  major 
characteristics  of  each  test  method  as  Well  as  the  quantitative  requirements 
which  must  be  specified  for  each  test  method.  The  data  analysis  method 
included  with  each  test  method  provides  the  decision  criteria  for  acceptance 
or  rejection  of  the  item  being  demonstrated. 

B.IO. 3  Sample  Size  -  Each  of  the  test  plans  contained  in  this  appendix  includes 
an  equation  or  other  directions  for  determining  a  minimum  sample  size  of  mainte¬ 
nance  tasks.  Any  departure  from  the  minimum  sample  size  requirement  can  affect 
the  statistical  validity  of  the  test  procedures.  Some  of  the  test  plans  in 
the  appendix  require  a  prior  estimate  of  the  variance  of  the  distribution  of 
interest  for  the  calculation  of  sample  size.  Such  prior  estimates,  subject  to 
government  approval,  can  be  obtained  from  data  on  similar  equiiment  provided 
similarities  in  maintainability  design,  skill  levels  of  maintenance  personnel, 
test  equijanent,  manuals  and  the  maintenance  environment  are  considered  in  the 
estimation  process.  Equations  for  predicting  the  variance  when  prior  estimates 
are  not  available  are  presented  in  DDC  document  AD-869396,  Maintainability 
Prediction  and  Demonstration  Techniques,  Vol.  II,  cited  in  para.  B.IO. 6,  which 
can  be  used,  provided  the  information  needed  for  the  prediction  is  available. 

The  85th  -  95th  upper  confidence  bound  on  the  predicted  or  estimated  variance 
shall  be  used  to  insure  preservation  of  the  desired  risk  values.  Average  observed 
values  of  the  variance  have  ranged  from  =  .5  to  =  1.3. 

B.IO, 4  Task  Selection  -  Selection  of  tasks  to  be  sampled  when  employing  fault 
simulation  will  be  made  in  accordance  with  Appendix  A  of  this  standard.  The 
Procuring  Activity  shall  have  the  option  of  surveillance  over  and/or  participat¬ 
ing  in  the  random  selection  of  tasks  comprising  the  demonstration  population 
(Column  11  of  Table  I)  down  to  and  including  the  specific  faults  to  be  simulated. 
This  shall  occur  at  a  specific  conference  at  a  time  established  by  the  contractor, 
consistent  with  the  Maintainability  Program  Plan  schedule.  In  the  event  that 
tasks  so  chosen  can  result  in  events  detrimental  to  safety  of  personnel  or 
property,  appropriate  redesign  action  must  take  place;  in  the  event  that  secondary 
failures  result,  they  will  be  documented  and  their  iii5)act  on  item  maintainability 
assessed.  A  report  of  such  findings  shall  be  made  to  the  procuring  activity. 

Care  must  be  exercised  in  selecting  and  sampling  tasks  to  insure  that  a  true 
simple  random  sample  is  obtained  when  sequential  tests  are  employed.  Departures 
from  simple  random  sampling,  such  as  proportionate  stratified  sampling,  can 
effect  the  validity  of  the  test  procedures  presented  herein,  however,  this  effect 
is  considered  minimal  for  the  sample  sizes  required  by  the  test  procedures  which 
are  not  sequential  tests.  Simple  random  sampling  shall  be  used  for  sequential 
tests. 

Supersedes  page  26  of  27  Iterch  1973 
26 


MIL-STD-471A 

10  Jan  1975 


B.10.5  Test  Selection  -  In  general,  the  test  index  to  be  demonstrated 
is  the  primary  consideration  in  selecting  a  test  procedure.  Considerable 
savings  in  sample  size  can  be  obtained  by  use  of  sequential  test  procedures 
in  preference  to  fixed  sample  tests.  As  a  general  rule,  however,  the 
sequential  test  should  be  used  only  when  prior  knowledge  (e.g.,  from  the 
prediction)  indicates  that  the  equipment  may  be  much  better  (or  worse) 
than  the  specified  Values. 

B. 10. 5.1  A  cross-reference  of  the  test  methods  in  MIL-STD-471A  to  those 
which  were  numbered  differently  in  the  original  issues,  MIL-STD-471 
(and  Notice  1)  are  listed  in  Table  lA. 


TABLE  lA,  TEST  METHOD  CROSS-REFERENCE  LIST 


MIL-STD-471A 
Test  Method  8 
Test  Method  9 
Test  Method  4 


Test  Method  10 
Test  Method  11- 


MIL-STD-471 


->  Test  Method  1 


Test  Method  2 


Test  Method  3 


■>  Test  Method  4 


Test  Method  6 
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The  justification  for  use  of  the  log-normal  assun5)tion  for  corrective 
maintenance  times  is  based  on  extensive  analysis  of  field  data  whi^  have 
shown  that  the  log-normal  distributicm  provides  a  good  fit  to  the  data. 
However^  in  those  cases  where  it  is  suspected  that  the  log-normal  assump¬ 
tion  does  not  hold  (e.g.,  equipments  with  a  high  degree  of  built-in 
diagnostics)  then  a  distribution- free  method  should  be  employed  to  in¬ 
sure  preservation  of  specified  risks. 

B.10.6  References  -  Details  and  additional  references  for  the  test  plans 
2,  3)  presented  in  this  appendix  can  be  found  in  RADC  Technical  Report 
69-356  (AD  869  396),  Volume  II,  entitled:  "Maintainability  Prediction 
and  Demonstration  Techniques."  Copies  of  this  document  may  be  obtained 
from  the  Defense  Documentation  Center,  Cameron  Statiem,  Alexandria,  VA 
22314. 


B.10.7  List  of  Symbols  -  The  following  symbols  and  notatiems  are  common 
the  test  methods  1-3  contained  in  this  appendix: 


X  ”  the  randexQ  variable  which  denotes  the  maintenance  characteristics 
of  interest  (e.g.,  X  can  denote  corrective  maintenance  time,  preventive 
maintenance  time,  fault  locati<Mi  tii^,  manhours  per  maintenmice  task, 
etc.) . 

Xi  ■  the  ith  observation  or  value  of  the  random  variable  X. 


n  B  the  sample  size 

1  ” 

r  »  the  sample  mean  (i.e.,  T  » 


(Xi) 


€*2 


n(inx-e){] 


i«l 


■  the  tmie  variance  of  InX 


E(X)  »  the  tnje  mean  of  X. 
d^  *»  Var(X)  ■  e[^->V)^Q  ■  the  true  variance  of  X, 

s  the  sample  variance  of  X  (i.e.,  ^  =  n=T  (xi-I^))= 

hit  Xl^-nT^) 


=  the  prior  estimate  of  the  variance  of  the  maintenance  time 

X  -  the  (l-p)th  percentile  of  X  (i.e.,  X.05  -  95th  percentile  of  X). 

M  «  X  *  the  median  of  X. 

y  ■  In  X  =  the  natural  logarithm  of  X. 

Y  •  the  sample  mean  of  Y 

0  -  E(ln  X)  ••  the  true  mean  of  In  X. 

^  ^  ■  the  prior  estimate  of  the  variance  of  the  logarithm  of  main¬ 
tenance  tims 
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2 

s  =  the  sample  variance  of  In  X. 

Zjj  =  the  standardized  normal  deviate  exceeded  with  probability  p 


1 

^  dz  =  p 

Ztx  ^(1-^)  ”  standardized  normal  deviate  exceeded  with 
probabilities  c<  and  (1--P)  respectively, 

«=<  =  the  producer's  risk,  the  probability  that  the  equipment  will 
be  rejected  when  it  has  a  true  value  equal  to  the  desired 
value  (Hq). 

p  =  the  consumer's  risk;  the  probability  that  the  equipment  will  be 
accepted  when  it  has  a  true  value  equal  to  the  maximum  tolerable 
value  (Hj), 

=  the  desired  value  specified  in  the  contract  or  specification 
and  is  expressed  as  a  mean,  critical  percentile,  critical 
maintenance  time. 

■  the  maximum  tolerable  value.  Note: 

IVhen  X  is  a  log-normally  distributed  random  variable: 

■f-M  ^  (j^  x  -  j 

If  y  «  In  X,  the  probability  density  of  Y  is  normal  with  mean  9  and 
variance 

y  ^  A'{ej 


Properties  of  the  log-normal  distribution: 


mean  « 

JL  ^  * 

variance  & 

d  -- 

JZ  (^  -  1 

median  « 

A1  = 

(<9  -  ) 

mode  B 

/M  = 

(l-p)th  percentile  * 

Xp  =•  ■2- 
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Table  of  standardized  normal  deviates: 

^  iiL 


.01 

.05 

.10 

.15 

.20 


2.33 

1.65 

1.28 

1.04 

.84 


.30 


.52 


B.10.8  List  of  Synbnls.  The  follrwing  syirbols  are  cermon  to  the  test  methods 
4,  8  -  11  contained  in  this  Appendix: 

X  .  =  'lamtenance  downtime  per  corrective  maintenance  task  (of  the 
ith  taslc ) . 

X  .=  'laintenance  downtime  per  preventive  maintenance  task  (of  the 
task). 

n  =  Nimiber  of  corrective  maint«iance  tasks  sampled, 
c 

*  Nun4)er  of  preventive  maintenance  tasks  sampled. 

P  “  Consuner's  risk. 

■  That  value,  corresponding  to  risk,  which  is  obtained  from 
a  t^le  of  normal  distribution  for  a  one-tail  test. 

f  =  Number  of  expected  corrective  maintenance  tasks  occurring 
during  a  representative  operating  time  (T), 

f  “  Nxiriber  of  expected  preventive  maintenance  tasks  occurring 
during  a  representative  operating  time  (T). 

T  ■  Item  representative  operating  time  period. 

Dt  ■  Total  maintenance  downtime  in  the  represeitative  operating 
time  (T) . 


~^/c 


■  I'iean  dowitimes  of  sanple.  (Corrective,  Preventive,  and  confcined 
Corrective /Preventive  "'laintenance  Times.) 


■  Sanple  calculated  maximum  corrective  maintenance  downtime. 


/Yc  •  Specified  mean  corrective  mainteaiance  time. 


y</pm  “  Specified  mean  preventive  maintenance  time. 
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^p/c  “  Specified  mean  maintenance  time.  (Taking  both  corrective 
and  preventive  maintenance  time  into  accoimt.  ) 

Mjnax  ”  -A-  requirement  levied  in  terms  of  a  maximum  value  of  a 
percentile  of  task  times  (i.e.,  95^  of  all  corrective 
task  times  must  be  less  than  60  minutes )  usually  taken 
as  the  90th  or  95th  percentile. 

J^axc  ~  Specified  corrective  maintenance  downtimes. 

^ax  “  Specified  preventive  maintenance  downtimes. 

=  E(  In  Xg )  =  Expected  value  of  the  logarithms  of  corrective 
maintenance  tasks. 

Log  Xci,  Log  Xq  =  Log  to  the  base  10  of  X^. 

In  Xgi,  InXc  =  Natural  logs  of  X^,  Xq. 

=  Median  value  of  corrective  maintenance  tasks. 

=  Median  value  of  preventive  maintenance  tasks. 
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TEST  METHOD  1 
TEST  ON  THE  MEAN 


B.20  General ^ -  This  test  provides  for  the  demonstration  of  maintainability 
when  the  requirement  is  stated  in  terms  of  both  a  required  mean  value  (iJ-i  ) 
and  a  design  goal  value  (Mq)  (or  when  the  requirement  is  stated  in  terms^of 
a  required  mean  value  ( ^  design  goal  value  ( Pq  )  is  chosen  by  the 
contractor).  The  test  plan  is  subdivided  into  two  basic  procedures,  identified 
herein  as  Test  Plan  A  and  Test  Plan  B.  Test  A  makes  use  of  the  lognormal 
assumption  for  determining  the  sample  size,  whereas  Test  B  does  not.  Both 
tests  are  fixed  sample  tests,  (minimum  sample  size  of  30),  which  employ  the 
Central  Limit  Theorem  and  the  asymptotic  normality  of  the  sample  mean  for 
their  development. 

Assumptions  -  Test  A  -  Maintenance  times  can  be  adequately  described 
by  a  lognormal  distribution.  The  variance,  of  the  logarithms  of  the 
maintenance  times  is  known  from  prior  information  or  reasonably  precise 
estimates  can  be  obtained.  Test  B  -  No  specific  assuii5)tion  concerning  the 
distribution  of  maintenance  times  are  necessary.  The  variance  d^  of  the 
maintenance  times  is  known  from  prior  information  or  reasonably  precise 
estimates  can  be  obtained . 


B.20. 2  Hypotheses  -  Hq:  Mean  =  Mq  (1-1) 

H^:  Mean  =  (yi>yo)  (1-2) 

Illustration;  Hq:  Pq  =  30  min. 

%:  Pi  =  45  min. 

Sample  Size  -  For  a  test  with  producer's  risk  a  and  consumer's 
risk  B,  the  sample  size  for  Test  A  is  given  by: 


- 1) 


(1-3) 


n  =  ZgWi)' 

(Nl  -  Pq)^ 

p 

where  a  is  a  prior  estimate  of  the  variance  of  the  logarithms  of  maintenance 
times.  The  sample  size  for  Test  B  is  given  by: 


n  = 


(1-4) 


^2  .. 


where  d  is  a  prior  estimate  of  the  variance  of  the  maintenance  times, 
and  Zg  are  standardized  normal  deviates. 

B.20.4  Decision  Procedure.  Obtain  a  random  sample  of  n  maintenance  times, 
^1’  ^2'  snd  compute  the  sample  mean. 


n 


x  =  -  Z 


_  1 


i=l 


^i 


(1-5) 
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and  the  sample  variance 


(^.1  - 


-  n  X 


Test  A;  Accept  if  7  ^  Z  e»<. 


Test  B;  Accept 


Lfl  < 


Cl-6) 


(1-7) 

(1-8) 


Reject  otherwise. 


B.20.5  Discussion  -  By  the  central  limit  theorem,  the  sample  mean  7 
is  approximately  normal  for  large  n  with  mean  E(X)  ^<1  variance  Var  (7). 
In  Test  A  un^r  the  log-noraal  assumption  Var  7  *  ^  where  d  « 
e(28-+'e^)  (e®^  -1)  =^^(6  -1)  thus  the  sarnie  size^can  be  computed  using 

a  prior  estimate  of^2,  in  Test  B,  a  prior  estimate  of  d^  is  assumed  to 
be  available  to  calculate  the  sample  size,  A.  critical  value  C  is  chosen 
such  that^o-^  Z^jvar  7*  C  -  Z^ar  7.  If.^  =  .  Then 

P  (X>C)  »o(and  if  =>*(],  then  P  (7-<C)  *  (9 

B.20.6  Example  -  It  is  desired  to  test  the  hypothesis  that  the  mean 
corrective  maintenance  time  is  equal  to  30  minutes  against  the  alternate 
hypothesis  that  the  mean  is  45  minutes  o<rP=^.05. 

Then  W^'.  Mo  “  ^0  minutes. 

-Ml  *  minutes. 

Test  A:  Under  the  log-normnl  assumption  with  prior  estimate  of&  =  .o^ 
the  sample  size  using  equation  1-3  is:  n  »  [1.65^30])  *  1«65(4^J_ 

(e-‘  -1)  .  S6 


(  ^jr-3o) 


Test  B:  Under  the  distribution-free  case  with  a  prior  estimate  of  d  ■ 

900,  (or  d  -  30),  the  sample  size  using  equation  1-4  is: 

nc“  =  43 

B.20.7  O.C.  Curve  -  The  OC  curve  for  Test  B  for  this  example  is  given 
in  Figure  H-T. — It  gives  the  pr<*ability  of  acceptance  for  values  of  the 
mean  maintenance  time  from  20  to  60. minutes.  The  (X  curve  for  Tes t^ _ A  for 
this  exanrole  is  given  in  Figure  B-2.  It  gives  the  probability  of  acceptance 
for  various  values  of  the  mean  maintenance  time.  Thus,  if  the  true  value 
ofJi  is  40  minutes,  then  the  prob^ility  tliat  a  demonstration  will  end  in 
acceptance  is  0.21  as  seen  from  Fig.  B-2. 
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TEST  METHOD  2 


TEST  ON  CRITICAL  PERCEMTILE 

B.30  General  -  This  test  provides  for  the  deaionstratlon  of  Balntalnahlllty 
when  the  requirement  is  stated  in  tezms  of  both  a  reqtilred  critical 
percentile  value  (Tjl)  and  a  design  goal  value  (Tq)  Cor  when  the  requlremait 
Is  stated  in  terms  of  a  required  percentile  value  (T^)  and  a  design  goal 
value  (Tq)  Is  chosen  by  the  contractor].  If  the  critical  percentile  Is  set 
at  ^0  percent,  then  this  test  method  Is  a  test  of  the  median.  The  test  Is  a 
fixed  sample  size  test.  The  decision  criterion  Id  based  upon  the  asymptotic 
normality  of  the  mftYinnaB  likelihood  estimate  of  the  percentile  value. 


£.30.1  Assumptlcai  -  Maintenance  times  can  be  adequately  described  by 
a  log>nozmal  distribution.  The  variance^ 2  of  the  loganHims  of  the 
maintenance  times  Is  known  frm  prior  InformatlcmL  or  reasc»iably  precise 
estimates  can  be  obtained. 


B.30.2  Hypotheses  -  Hq:  (l-p)th  percentile,  Xp  *  Tq  (2-1) 

or  P  tx>ro^  =  p 

Hi:  (l-p)th  percentile,  Xp  =  T^^  (2-S) 

or  P  ^  X>T;^  »  Pf  (Ti>To) 

Illustration:  Hq;  95th  percentile  »  3^  =  X^qj  “  1»5  hours  * 

Tq:  inTo  “  .1»055 

Hi:  95th  percentile  “  ^  ■  X  q^  «  2  hours  * 

Ti:  InTi  »  .6932 


B.30.3  Sample  Size  -  To  meet  specif led*<and  ^  risks,  the  sample  size 
to  be  used  is  given  by  the  formula 

2\  Zfi 


'^2  1  \  (Round  iqp  to  next  integer) 

InT,  -  in!P„i  (2-3) 


where  « 

2^2  Is  a  prior  estimate  of^,  the  true  variance  of  the  logarltlmB  of  the 
maintenance  times . 


Zp  Is  the  standardized  normal  deviate  corresponding  to  the  (l  -  p)th 
percentile . 


B.30.4  Decision  Procedure  -  CoBqnite 


Y  e 


1 

n 


s^ 


»  1 
n-1 


(2-if) 

(2-5) 
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1/2 


InT^  +  Z^S 


1  Z_^ 

—  +  p 

"  Tunr 


Accept 
Reject  otherwise. 


if  7”  +  Z  S  £  X‘ 
P 


(2-6) 

(2-7) 


B.30.5  Discussion  -  This/test  is  based  upon  the  fact  that  under  the 
log-normal  assumption,  the  (l-p)th  percentile  value  is  given  by  X  =  e^®  * 

Taking  logarithms  gives  In  Xp  -  9  +  zC,  and  using  maximum  likelihood 

estimates  for  the  normal  parameters  g^andiS-,  the  (l-p)th  percentile 

maximum  likelihood  estimate  is  t  c  nv-i  \  ^  , 

InXp  n  Y  *  Z  SWEli^lnXp  is  approximately 

criJfhi  producer’s  and  consumer's  riSk  requirements,  a 

^“5  I  <^osen  for  the  samj>le  estimate  of  the  (l-p)th  per¬ 
centile  Xp.  Note  Y  =  9  an  estimate  for  9  .  ^  I'J  f 

B.30.6  Example  -  The  following  hypotheses  are  to  be  tested  .10 

Hq;  95th  percentile  -  X^qs  =  1*5  hours  =  Tq;  InT©  =  .4055 

Hj;  95th  percentile  =  X  qs  ■  2.0  hours  =  Tj;  InTi  =  .6932 

A  prior  estimate  of  is  equal  to  1,0  using  equation  2-3. 

(1.63)^"^  (1)  (2.56)^ _ _ 


nc=^^ 


(In  2.0  -  In  1.5)‘ 


or 


ng=  187 

The  critical  value  jj*  is  given  by  equation  2-5 


or 

X*  «.4055  0.1437S 


B.30.7  PC  Curve  -  The  OC  curve  for  Test  Method  2  for  this  example 
is  pven  in  Figure  B-4.  It  gives  the  probability  of  acceptance  for 
various  values  of  the  95th  percentile  of  the  maintenance  time  distribu- 

^0  05  hours,  then  the  probability 

that  a  demonstration  will  end  in  acceptance  is  0.57  as  seen  from 
Figure  B-4. 
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TEST  ON  CRITICAL  MAINTENANCE  TIME  OR  MANHOURS 


B.40  General  -  This  test  provides  for  the  demonstration  of  maintainability 
when  the  requirement  is  specified  in  terms  of  both  a  required  critical 
maintenance  time  (or  critical  manhours)  (Xp^)  and  a  design  goal  value  (Xp  ) 

(or  when  the  requirement  is  stated  in  terms  of  a  required  critical  maintenance 
time  (Xp^)  and  a  design  goal  value  (X^)  is  chosen  by  the  contractor).  The 

test  is  distribution-free  and  is  applicable  when  it  is  desired  to  establish 
controls  on  a  critical  upper  value  on  the  time  or  manhours  to  perform  specific 
maintenance  tasks.  In  this  test  both  the  null  and  alternate  hypothesis  refer 
to  a  fixed  time  and  the  percentile  varies.  It  is  different  from  Test  Method  2 
where  the  percentile  value  remains  fixed  and  the  time  varies. 

B.40.1  Assumptions  -  No  specific  assumption  is  necessary  concerning  the 
distribution  of  maintenance  time  or  manhours. 


B.40.2  Hypothesis  -  Hq:  T  =  Xp  (3-1) 

°  (Pl>Po) 

%:  T  =  Xp^  (3-2) 

For  specified  a  and  3: 

Illustration  -  HqI  30  min.  “  Xq  =  50th  percentile  (median) 

%:  30  min.  =  Xq  -75  =  25th  percentile 

B.40.3  Sample  Size,  n,  and  Acceptance  Number,  c  -  The  normal  approximation 
to  the  binomial  distribution  is  employed  to  find  n  and  c  when  pq  is  not  a 
small  value.  Otherwise,  the  Poisson  approximation  is  employed.  The 
equations  for  n  and  c  are  as  follows: 


For  0.20  <Pq<0.80  (pj  =  1  -  Qj^) 

:2 


=  ^  Zojp^ 


2 

n  =  _ 


I 


c  =  n 


Pi  -  Po 
Z3Pot^  ^  Zapjp^ 


- 


Zo^iPo^o  * 


<^1^1 


(Use  next  higher 
integer  value. ) 
(3-3) 

(Use  next  lower 
integer  value. ) 
(3-4) 


For  Pq<0.20 


For  this  case  n  and  c  can  be  found  from  the  following  two  equations: 
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f 

o 


'c-  V,  , 

'Trl  2.  ' 


o 


y  e  ('”/’/)^  <  3 

■“  — /  “* 


(3-5) 


(3-6) 


Table  B-I  provides  sampling  plans  for  various  <X-and  /^xisks  and 
ratios  pj^/p^when  p^<0,20. 


B.40,4  Decision  Procedure.  Random  samples  of  maintenance  times  are 
taken,  yielding  n  observations  Xi,  X2,  X  .  The  number  of  such 

observations  exceeding  the  specified  time  T  is  counted.  This  number  is 
called  r, 


Accept  Hg  if  r<c,  (3-7) 

Reject  Hq  if  r  >  c.  (3-8) 

B.40,5  Discussion.  In  the  development  of  the  decision  criteria  and 
sample  size,  equations  for  this  tesVthe  normal  or  Poisson  approxima¬ 
tion  to  the  binomial  distribution  is  lised. 

B.40,6  Example.  A  median  value  of  30  minutes  is  considered  acceptable 
whereas  ii  30  minutes  is  the  25th  percentile  then  this  is  considered 
vinacceptab le .  The  following  hypotheses  result:  (o^=(3=  .10) 

Hg:  30  minutes  *  Xg^5g  =  50th  percentile  median 

Hj:  30  minutes  »  Xg^75  *  25th  percentile 

Then  Zfi  =  1.28,  Po  =  .50,  =  .75  using  equations  3-3  8  3-4. 


n 

and. 


c 

B.40.7  OC  Curve  -  The  OC  curve  for  Test  Method  3  for  this  example  is 
given  in  Figure  B-5.  It  gives  the  probability  oi^  acceptance  for  values 
of  probability  p,  varying  from  0.3  to  1.0.  Here  Xp  is  (1-p)  th  percentile. 
Thus,  if  the  true  value  of  the  given  critical  maintenance  time  is  40th 
percentile,  i.e.,  if  the  value  of  p  is  0.6,  then  the  prc*ability  that  a 
demonstration  will  end  in  acceptance  is  0.61  as  seen  from  Fig,  B-5, 


(1.28)' 


\/(,75)(,25)  +y(.50)(.50) 

7755) 


i 


23 


23 


:i.28).5  \l  (,75)(,25)  +  1,28  (.75)  \/(.50)  (._50) 

'l.V  V 


T-.14 
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TEST  ON  THE  MEDIAN  (ERP) 

B.50  General  -  This  method  provides  for  demonstration  of  maintain¬ 
ability  when  the  requirement  Is  stated  in  terms  of  an  equipment 
repair  time  (ERT)  median,  which  will  be  specified  In  the  detailed 
equipment  specification. 


Assumption  -  This  method  assumes  the  underlying  distribution  of 
corrective  maintenance  task  times  Is  lognormal. 

®*50.2  Sample  Size  -  The  sample  size  required  Is  20.  This  sample  size 
must  be  used  to  employ  the  equation  described  In  this  test  method. 


B.50.3  Task  Selection  and  Performance  -  Sample  tasks  shall  be  selected 
in  accordance  with  the  procedure  outlined  in  Appendix  "A” .  The  duration 
of  each  shall  be  recorded  and  used  to  conq)Ute  the  following  statistics: 


nc 

Log  ^  (Log  Xci) 

1-1 


fttc 

(logX 

1-1  _ 


Cl 


-  (log  Mm^)' 


All 

logarithim 
will  be 
taken 
to  the 


(4-2) 


base  10 


*  nc  J 

Where:  MPTEq  Is  the  m^ured  geometric  mean  time  to  repair.  It  Is 
the  equivalent  to  the  M,.^  used  in  other  plans  Included  In  this  document. 


B*50.^  Decision  Procedure  -  The  equipment  under  test  will  be  considered 
to  have  met  the  maintainability  requirement  (ERT)  when  the  measured 
geometric  mean -time -to-repair  (MUJTHq)  and  standard  devlatlon(S)  as  determined 
in  50.3  satisfies  the  following  expression: 


Accept  If  log  MTTHs^log  ERT  +  .397(S)  (L_3) 

where:  log  ERT  -  logarithm  of  the  equipment  repair  t.iinp! 

log  Mn?RQ  -  the  value  determined  in  accordance  with  para.  50. 3 
S  -  the  value  determined  in  accordance  with  para.  50.3 

®*50.5  Discussion  -  The  value  of  equipment  repair  time  (ERT)  to  be 
specified  in  the  detailed  equipment  specification  should  be  determined 
using  the  follovring  expression: 

ERT  (specified)  -0.37  EKPnax 
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where: 


ERTjji^  =  the  maximum  value  of  ERT  that  should  he  accepted  no 
more  than  10  percent  of  the  time. 

0.37  =  a  value  resulting  from  application  of  "student's  t" 

operating  characteristics  that  assures  a  95  percent 
probability  that  an  equipment  having  an  acceptable 
ERT  will  not  be  rejected  as  a  result  of  the  maintain¬ 
ability  test  when  the  sample  size  is  20,  and  assuming 
a  population  standard  deviation  (d)  of  0.55. 


B. 50. 5.1  Derivation  of  Criteria  -  The  following  are  brief  explana¬ 
tions  of  the  derivations  of  various  criteria  specified  herein,  and 
are  intended  for  information  purposes  only.  The  acceptance 
criterion,  log  MTlRQ^-log  ERT  /  0.397(S),  assures  a  probability  of 
.95  of  accepting  an  equipment  or  system  as  a  result  of  one  test 
when  the  true  geometric  mean-time- to-repair  is  equal  to  the  specified 
equipment  repair  time  (that  is,  a  probability  of  0.05  of  rejecting  an 
equipment  or  systems  having  a  true  MTTRq  equal  to  the  specified  ERT). 
This  was  derived  by  using  conventional  methods  for  establishing 
acceptance  criteria.  The  conventional  methods  for  determining 
acceptance  based  on  the  measured  mean  of  a  small  sample  (that  is, 
sample  size  less  than  30),  and  when  the  true  standard  deviation  (d) 
of  the  population  can  only  be  estimated,  is  to  compare  the  measured 
mean  with  the  desired  mean  using  the  expression: 


where : 


S 


the  standard  deviation  of  the  sample; 


X  =  the  sample  or  measured  mean 
Xq  =  the  specified  or  desired  mean 
n^  =  the  sample  size 

x^  =  the  value  of  one  measurement  of  the  sample 


The  decision  to  accept  the  product  will  be  made  when  the  test  results 
give  a  value  of  t,  as  calculated  from  the  above  expression  numerically 
less  than  or  equal  to  a  value  of  t  obtained  from  "student's  t'.' 
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distribution  tables  at  the  established  level  (that  is,  0.99,  0.95, 
0.90,  and  so  forth)  of  acceptance  and  the  appropriate  sample  size. 
The  "student's  t"  distribution  tables  (for  a  single  tailed  area) 
give  a  value  of  t  =  1.729  at  the  0.95  acceptance  level  when  the 
SEimple  size  is  20  (that  is,  19  degrees  of  freedom).  The  table  for 
single  tailed  area  is  used  since  only  values  of  MTTRq  greater  than 
the  specified  ERT  are  critical.  An  equipment  with  any  value  of 
MTTRg  lower  than  the  specified  ERT  is  acceptable.  To_  apply  the 
expression  for  "t"  to  the  maintainability  test,  let  Xq  =  log  ERT 
(specified),  x  =  log  MTTRg  (measured),  S  =  the  measured  standard 
deviation  of  the  logarithms  of  the  sample  of  measured  repair  time, 
and  n^  =  the  sample  size  of  20.  The  measured  MTTRq  is  then  compared 
with  the  desired  ERT  by  calculating  the  value  of  t  using  the  expres¬ 
sion  below: 


^  ^  (log  MTTRg  -  log  ERT) 
S 


The  equipment  ;inder  test  can  be  acceptable  if  the  value  of  t  calculated 
from  the  expression  above  is  equal  to  or  less  than  /  1.729  (the  value 
of  t  from  the  "student's  t"  distribution  tables  at  an  acceptable  level 
of  .95  when  the  sample  size  is  20).  Therefore,  the  equipment  should 
be  accepted  when: 

(log  MTTRg  -  log  ERT)  ^  ^  1  70Q 

S  "  *  ■ 

Upon  rearranging  and  simplifying  this  expression,  the  acceptance 
criterion  is  obtained  as  shown  below: 


log  MTTRg  - 


log  ERT  f 


1.729(S) 

sji^ 


log  MTTRQ^log  ERT  /  .397(S) 

(NOTE:  Reference  -  "Introduction  to  Mathematical  Statistics," 

P.  Hoel,  J.  Wiley  and  Sons,  Inc.,  2nd  Edition,  1954,  PP-  222-229) 
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TEST  METHOD  5 

TEST  ON  CHARGEABLE  MAINTENANCE  DOWNTIME  PER  FLIGHT 


B,60  General  -  Because  of  the  relatively  small  size  of  the 
demonstration  fleet  of  aircraft  and  administrative  and  operational 
differences  between  it  and  fully  operational  units,  operational 
ready  rate  or  availability  cannot  be  demonstrated  directly. 

However,  a  contractual  requirement  for  chargeable  downtime  per 
flight  can  be  derived  analytically  from  an  operational  requirement 
of  operational  ready  rate  or  availability.  This  chargeable  downtime 
per  flight  can  be  thought  of  as  the  allowable  time  (hours)  for 
performing  maintenance  given  that  the  aircraft  has  levied  on  it  a 
certain  availability  or  operational  readiness  requirement.  The 
requirement  for  chargeable  downtime  per  fligjit  will  be  established 
using  the  procedure  in  B.60.3.  Chargeable  downtime  per  fli^t 
can  then  be  demonstrated  using  the  procedures  in  B.60.5. 

B.60.1  Definitions  -  The  following  definitions  apply  to  this  test 
method: 

A  s  Availability  -  A  measure  of  the  degree  (expressed  as  a 
probability)  to  which  an  aircraft  is  in  the  operable  and  committable 
state  at  the  start  of  the  mission,  when  the  mission  is  called  for  at 
an  unknown  (random)  point  in  time.  In  this  standard,  availability 
is  considered  synonomous  with  operational  readiness.  The  aircraft  is 
not  considered  to  be  in  an  operable  and  committable  state  when  it  is 
being  serviced  and  is  undergoing  maintenance  (see  MIL-STD-721B) . 

TOT  *  Total  Active  Time  in  Hours. 

Active  Time  =  That  time  during  which  an  aircraft  is  assigned  to 
an  organization  for  the  purpose  of  performing  the  organizational 
mission.  It  is  tin«  during  which: 

1.  The  aircraft  is  flying  or  ready  to  fly. 

2.  Maintenance  is  being  performed. 

3.  Maintenance  is  delayed  for  supply  or  administrative  reasons. 

DUR  »  Daily  Utilization  Rate  -  The  number  of  flying  hours  per  day. 

AFL  =  Average  Flight  Length  -  Flying  h<nirs  per  flight, 

NOF  »  Number  of  Flights  per  Day. 
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DT  =  Downtime  -  Time  (in  hours)  during  which  the  aircraft  is  not 
ready  to  commence  an  assigned  mission  (i.e.,  have  the  flight  crew 
board  the  aircraft). 

CMDT  =  Chargeable  ‘laintenance  Downtime  -  Time  (in  hours)  during 
which  maintenance  personnel  are  working  on  the  aircraft,  except  when 
the  only  work  being  done  would  fall  under  the  nonchargeable  maintenance 
downtime  (NCMDT)  category. 


NCflDT  =  Nonchargeable  Maintenance  Downtime  -  Time  (in  hours)  during 
which  the  aircraft  is  not  available  for  immediate  flight  but  the  only 
maintenance  being  performed  is  not  chargeable.  It  would  include  the 
following: 


'1.  To  correct  maintenance  or  operational  errors  not 
attributable  to  technical  orders,  contractor  furnished  training  or 
faulty  design. 


2.  Miscellaneous  tasks  such  as  keeping'of  records  or  taxiing 
or  towing  the  aircraft  to  or  from  other  than  the  work  center  area. 

3.  Repair  of  accident  or  battle  damage. 

4.  Modification  tasks. 

5.  Maintenance  caused  by  test  instrumentation. 

DDT  =  Delay  Downtime  -  Downtime  (in  hours)  during  which  maintenance 
is  required  but  no  maintenance  is  being  performed  on  the  aircraft  for 
supply  or  administrative  reasons.  It  would  include  the  following: 

1.  Supply  Delay  Downtime. 

a.  Not  Operationally  Ready  Supply  (NORS)  time. 

b.  Item  obtainment  time  from  other  than  the  work  center 

area. 

2.  Administrative  Delay  Downtime. 

a.  Personal  breaks  such  as  coffee  or  lunch. 

b.  No  maintenance  people  available  for  administrative 

reasons . 
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«<  =  the  producer’s  risk:  The  risk  that  the  producer  (contractor) 

must  take  that  the  hypothesis  that  a  true  mean  =  Mo  rejected 

even  though  it  is  true.  Tlie  desirable  value  of  must  be  determined  by 
judgement  and  agreed  upon  by  the  procuring  activity  and  the  contractor. 
All  other  things  being  equal,  a  smaller  value  of will  require  a 
larger  sample  size. 

*1  =  TTie  maximum  mean  chargeable  maintenance  downtime  per  flight. 

Mq  =  TTie  required  mean  C^^DT  per  flight. 

M_Mq  s  TTie  difference  between  the  maximum  mean  (M)  of  the  parameter 
being  tested  and  the  specified  mean  (Mq).  This  value  must  be  determined 
in  conjunction  with  a  value  for/3,  the  consider 's  risk.  M  is  a  value, 
greater  (worse)  than  the  specified  mean,  which  the  consun»r  is  willing  to 
accept,  but  only  with  a  small  risk  or  probability  O).  If  the  true  mean 
is  in  fact  equal  to  the  value  of  M  selected,  the  hypothesis  the  true 
mean  =  Mq  will  be  accepted,  although  erroneously,  100  p  percent  of  the 
time. 

=  the  consumer's  risk.  The  risk,  which  the  consumer  is  willing 
to  take,  of  accepting  the  hypothesis  that  the  true  mean  =  Mq  when  in 
fact  the  true  mean  ■  M.  All  other  things  being  equal,  a  smaller  value 
of P  will  require  a  larger  sample  size. 

^  =  the  true  standard  deviation  of  the  parameter  (CMDT  per  flight) 

being  tested.  This  value,  unless  it  is  a  specification  requirement, 
will  not  be  known,  but  an  estimate  must  be  made.  (It  is  assumed  that 
both  M  and  Mq  will  have  the  same  value  of <3“.)  The  contractor's  maintain¬ 
ability  math  model,  previous  models,  or  previous  data  may  be  used.  All 
other  things  being  equal  a  larger  value  of ^ will  require  a  larger 
sample  size. 

B.60.2  Assumptions  -  This  method  requires  no  assumption  as  to  the 
probability  distribution  of  chargeable  downtime  per  flight.  The  metiiod 
is  valid  only  if  the  Central  Limit  Theorem  applies,  which  means  that  the 
sample  size  (number  of  flights)  must  be  large  enough  for  this  theorem  to 
apply.  The  sample  size  shall  be  at  least  50,  but  the  actual  size  is  to 
be  determined  in  accordance  with  para.  B.60.4. 

B.60.3  Derivation  of  CMDT  per  Flight  frcrni  Av^lability.-  The  requirement 
for  CMDT  per  flight  which  will  be  demonstrated  will  be  determined  using 
the  following  mathematical  derivation. 
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(5-1) 


A  (TOT)  =  TOT  -  CMDT  -  NC'  IDT  -  DDT 

Cf  tDT  =  TOT  -  A  (TOT)  -  NCf  T>T  -  DDT 

01DT  -TOT  -  A (TOT)  -  NCMDT  -  DDT 
NOF  NOF 

but, 

NOF  =  HLim 
24  (AFL) 


(5-2) 

(5-3) 

(5-4) 


(5-5) 


therefore, 

=  24  (AFL)  _  A (24)  (AFL)  NCMDT  DDT 

NOF  DUR  DUR  ”  NOF  '  NOF  (5-6) 


CMDT 

NOF 


CMDT  per  flight,  which  will  be  demonstrated. 


Values  for  UR  and  AFL  should  be  those  planned  for  the  aircraft  during 

operational  me.  Values  for  and  are  a  function  of  the 

NOF  NOF 


operational  environment.  They  will  be  provided  to  the  contractor  in  the 
RFP  or,  if  not,  will  be  provided  by  him  in  his  proposal.  The  value 
or  availability  or  operational  ready  rate  will  be  provided  in  the  RFP. 


Example:  Following  is  an  example  of  how  a  requirement  for  CMDT  per 
flight  will  be  derived: 

Required  A  =  0.75 


DUR  =  2  hours  per  day 
AFL  =  4  hours  per  flight 
NCMDT 

”N(|)F  '  ®  0*2  hours  per  flight 
=  1.0  hours  per  flight 
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Then, 

CMPT  ,,  24(4)  .  (0.75)  (24)  (4)  .  o.2  -  1.0 
NOF  2  '  2 

=  48  _  36  -  0.2  -  1.0 

NOF 

=  10.8  hours  per  flight 


B.60.4  Sample  Size  -  Since  the  Central  Limit  Theorem  is  applied,  the 
expected  distribution  of  the  means  will  take  on  a  normal  distribution  ^ 
as  in  Figure  B-6.  If  the  true  mean  is  equal  to  Mp  a  particular  is 
desired  the  upper  distribution  (the  mean  of  the  distribution  will 
equal  Mq)  will  apply.  It  is  on  £his  basis  that  an  acceptance  rule  is 
generated  to  the  effect  that  if  X  is  found  to  be  equal  to  or  less  than 
the  value  Mf,  +  the  item  is  to  be  accepted. 

If  the  true  mean  is  equal  to  M  (which  is  greater  than  Mq)  the 
distribution  of  means  will  take  on  a  normal  distribution  with  a  mean 
of  M  as  shown  in  the  lower  distribution.  The  value  to  be  used  as  an 
acceptance  criterion  corresponds  and  is  equal  to  a  value. 

nTH" 

(M  +  )  where  is  a  new  confidence  level 


where 

Mo 


M  ^  M  +  2^ 

^  Nj-r 

(5-7) 

:  '4o  +  (M-Mo) 

(5-8) 

+  Mo  +  M-Mo  +  1=^ 

vTiT 

(5-9) 

or  simplifying,  the  sample  size  (n)  requirement  is: 

n  »  CZcK-  Zo<)^  (Zpc“  Z(i-^)^  (5-10) 

WT'  (^' 
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If  this  expression  should  result  in  n  less  than  50,  then  a 
sanple  of  50  shall  be  used. 

o<.  =  Prob.  of  rejection  if  true  nean  equals  N. 

1  -  t<'^=  3=  Prob.  of  acceptance  if  true  mean  equals  M, 

Zoo  =  standardized  normal  deviated  as  defined. 

See  table  below  for  relationships  between  Zw  and  o<§  (3 

w  =  «=><  or  1- 


7  .01  .05  .1  .15  .2  .3  .7  .8  .85  .9  .95 

^  2.33  1.65  1.28  1.04  .84  .52  -.52  -.84  -1.04  -1.28  -1.65  - 

=  Zo^or 

Example:  Suppose  for  a  reauirement  of  Mq  "  2.0,  the  following  statistical 
test  parameters  were  agreed  to  by  the  procuring  activity  and  the 
contractor: 

cx  *  0.10;  2ot=  1*28;  <3  =  0.10;  -9=  -1.28;  M-^Iq  =  0.30;  S'  =  1.0; 


0.3 


Using  equation  5-10: 

n  =  (1.28  -f  1.28j^  ^  (2.56)^  ^  6.57 
(.3)2  (.3)2  ,09 


73 


.99 

2.33 
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B.60.5  Decision  Procedure  -  The  chargeable  maintenance  downtime  (Xi) 
after  each  flight  will  be  measured  and,  at  the  end  of  the  test,  the 
total  chargeable  downtime  will  be  divided  by  the  total  number  of 
flights  to  obtain  (30  the  sample  mean  C'T)T  and  the  sample  standard 
deviation  (s)  of  CMDT, 


X -iil 


NOF 


(5-11) 


s  = 

\ 


-2 

H  (Xi  -  X) 
NOF-1 


'I 


NOF 


_  2 


(NOF-1)  i=l 


21  Xi^-(N0F)X 


Accept  if:  X  <  Mo 


Zo<S 


\|'nof 


Reject  if:  X  >  Mq 

MNOF 


(5-12) 

(5-13) 

(5-14) 
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TEST  ^lETHOD  6 
TEST  ON  MANHOUR  RATE 


B.70  General  -  This  test  for  demonstrating  manhour  rate  (manhours  per 
fli^t  hour)  is  based  on  a  determination  during  Phase  II  test  operation 
of  the  total  accumulative  chargeable  maintenance  manhours  and  the  total 
accumulative  demonstration  flight  hours.  The  demonstrated  manhour  rate 
is  calculated  as: 

Manhour  Rate  =  Chargeable  Maintenance  Manhours  ('6-1') 

Total  Demonstration  Fligtit  Hours  ^ 

If  the  demonstrated  manhour  rate  is  less  than  or  equal  to  the  manhour 
rate  requirement  plus  a  maximum  value  ^4MR) ,  by  which  the  demonstrated 
manhour  rate  will  be  permitted  to  differ  from  the  required  manhour 
rate,  then  the  requirement  has  been  met.  ^  MR  will  be  provided,  by  the 
procuring  activity,  as  a  percentage  of  the  system  manhour  rate  require¬ 
ment  and  will  be  determined  based  upon  such  considerations  as  the 
expected  Phase  II  duration,  and  prior  experience  with  similar  systems. 

It  is  recognized  that  this  demonstration  method  is  nonstatistical  in 
nature  and  does  not  allow  the  determination  of  quantitative  producer^’s 
and  consumer's  risk  levels.  It  is  for  this  reason  that  the/^MR  is 
provided  (in  a  subjective  manner)  to  minimize  the  producer's  risk. 

B.70.1  Normally,  all  maintenance  performed  by  approved  test  maintenance 
personnel  during  Phase  II  and  documented  in  appropriate  maintenance 
reports  will  be  the  source  of  data  for  identifying  chargeable  maintenance 
manhours.  The  procuring  activity  may  elect  to  terminate  the  demonstration 
prior  to  Phase  II  completion  if  sufficient  data  are  collected  to  project 
that  the  requirement  will  be  met. 


B.70. 2  The  manhour  rate  requirement  must  pertain  to  the  aircraft 
configuration  provided  for  in  the  contract.  For  Phase  II  flights 
conducted  with  a  configuration  other  than  this,  an  appropriate  amount  of 
chargeable  manhours  will  be  included  in  calculating  the  total  chargeable 
manhours.  This  amount  will  be  based  upon  the  predicted  manhour  rate 
associated  with  the  equipment  not  installed. 

B,70.3  Care  must  be  exercised  in  assuring  that  the  predicted  manhour 
rate  pertains  to  flight  time  and  not  equipment  operating  time.  The 
contractor  must  develop  appropriate  ratios  of  equipment  operating  time 
to  flight  time, 
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TEST  ON  MANHOUR  RATE  .  (USING  SlrOJLATED  FAULTS) 


B.80  General.  This  test  for  demonstrating  manhour  rate  (manhours 
per  operating  hour)  is  based  on  (a)  the  predicted  total  failure 
rate  of  the  equipment  used  in  the  formulation  of  Table  I,  Appendix  A, 
(b)  the  total  accumulative  chargeable  maintenance  manhours  and  the 
total  accumulative  simulated  demonstration  operating  hours.  The 
demonstrated  manhour  rate  is  calculated  as: 


Manhour  Rate 


T.  Xci  +  (PS) 

Total  Chargeable  Maintenance  Hours  i^l 

Total  Operating  Time  “  T 


(7-1) 


where: 


Xci  =  Manhours  for  corrective  maintenance  task  i 

n  =  Number  of  corrective  maintenance  tasks  sampled,  n  shall 
not  be  le$s  than  30 


MTBF  =  MTBF  of  the  unit  (value  used  in  development  of  Table  I) 

(PS)  =  Estimated  average  total  manhours  which  would  be  required  for 
preventive  maintenance  during  a  period  of  operating  time  equal 
to  n‘/tMTBF)  hours 


n  _ 

number  of  corrective  maintenance  manhours  per 
1=1”;^  corrective  maintenance  task 


T  =  Operating  time 


B.SO.l  Discussion. 
T-=  n‘ (MTB^)  where 

question. 


When  maintenance  tasks  are  simulated  as  in  Table  1, 
-i: _ =  the  total  failure  rate  of  the  equipment  in 


21  Xci  +  (PS)  t.  Xci  +  (PS) 

i  =  l _ s  i^l _ _1 [Xc  + 

T  n* (MTBF)  MTBF 

All  components  of  (7-2)  with  the  exception  of  IL  can  be  considered 
constants.  Xc  can  be  considered  a  normally  distributed  variable  when  n 
is  large  (due  to  the  Central  Limit  Theorem)  with  Variance  = 

n 

If  Tc  is  normally  distributed  it  can  be  shown  that  the  function: 
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1  —  PS 

('VttbfT  iT’J  also  normally  distributed  around  the 

1  d  ^ 

mean  of  the  manpower  rate  with  Variance  =  (“)  D  ; 

assuming  d  «  d. 

B.80.2  Decision  Procedure.  Therefore,  if  the  manhour  rate  requirement 


Accept  if: 


pc  j 

X.  <Vr  (MTBF)  -  C— )  * 

^  n 


(7-3) 


IVhereoc  denotes  producer's  risk. 
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TEST  ON  \  CCyiBINED  AN/PERCENTILE  REQUIREMENT 

B.90  General  -  This  test  provides  for  tiie  demonstration  of  maintainability 
when  the  specification  is  couched  in  tenas  of  a  dual  requirement  for  the 
mean  and  either  the  90th  or  95th  percentile  of  maintenance  times  when  the 
distribution  of  maintenance  time  is  lognormal. 

B,90,l  Assumptions  -  For  use  as  a  dual  mean  and  90th  or  95th  percentile 
requirement  the  mean  must  be  greater  then  10  and  less  than  100  units  of 
time;  the  ratio  of  the  90th  percentile  maximum  value  to  the  value  of  the  mean 
must  be  less  than  two;  the  ratio  of  the  95th  percentile  maximum  value  to 
the  value  of  tlie  mean  must  be  less  than  three. 

Maximum  Ratio  of  Percentile  to  Tfean 

9Qth  Percentile  Value  2 

95th  Percentile  Value  3 

Distribution  assumptions  are  as  defined  in  B.90. 

B.90. 2  Discussion  -  The  test  metiiod  actually  demonstrates  the  61st  percentile 
value  of  maintenance  time  in  contoination  with  either  the  90th  or  95th 
percentile  values  of  maintenance  time  rather  than  the  mean  value  of  maintenance 
time  in  combination  with  either  the  90th  or  95th  percentile  values  of  maintenance 
time.  However,  because  of  the  particular  characteristic  of  the  lognormal 
distribution  once  a  61st  percentile  value  of  maintenance  time  less  than  Xi  and 
a  90th  or  95th  percentile  value  less  than  X2  has  been  demonstrated,  for  all 
practical  purposes  a  mean  value  of  less  than  approximately  Xi  and  a  90th  or 
95th  percentile  value  less  than  X2  have  likevdse  been  demonstrated, 

A  dual  requirement  on  maintainability,  assuming  a  lognormal  distribution  of 
repair  times ,  of  a  maximum  value  of  the  Mean  in  conjunction  with  either  the 
maximum  value  of  the  90th  or  95th  percentile  of  repair  time  (to  be  referred 
to  as  results  in  the  definition  of  various  combinatiens  of©  *8  and 

which  are  acceptable  to  the  dual  requirement,  (A  con5)lete  technical  description 
of  a  lognormal  distribution  is  provided  by  knowledge  of  9  8  #7  henc:e,  all  possible 
lognormal  distributions  acceptable  to  the  requirements  are  defined  through 
definition  of  all  possible  acceptable  values  of  0  and®^)  See  Figure  B-8A 
wfiich  defines  the  acceptable  cenibinations  of  0  §  ^or  a  >fean  of  30  minutes  and 
a  95th  percentile  (\ax)  minutes. 

For  the  lognormal  distributiem  it  is  also  possible  to  structure  a  dual 
requirement  made  up  of  the  ■axlmi  values  of  two  percentiles  (for  example,  the 
61st  percentile  of  repair  time  shall  be  a  maximum  of  30  minutes  and  the 
95th  percentile  of  repair  time  shall  be  a  maximum  of  60  minutes) .  This 
dual  requirement  also  results  in  the  definition  of  various  combinations  of 
acceptable  values  of  0  andcT.  See  Figure  B-9B.  If  a  dual  percentile 
requirement  could  be  structured  such  that  the  set  of  acceptable  values  of 
9  and  (THef ined  were  almost  identic:al  to  the  set  of  values  of  0  andS^defined 
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for  a  given  dual  Mean  and  percentile  requirement  then  a  demonstration 
of  that  dual  percentile  requirement  would  in  reality  also  demonstrate 
the  attainment  of  the  dual  Mean  and  Htiay  requirement.  For  this 
particular  instance  it  has  been  found  that  under  the  assumptions  listed 
above,  almost  identical  acceptable  values  of  0  and  CTare  provided  for  a 
combined  Mean  and  S^iax  requirement  and  a  combined  61st  percentile  (where 

the  value  of  the  61st  percentile  is  taken  equal  to  the  specified  value  of  the 
Mean)  and  l^nax  requirement.  See  Figure  B-8B  which  defines  the  values  of 
6  and  ^ acceptable  to  a  dual  61st  percentile  (where  the  value  of  the  61st 
percentile  is  taken  equal  to  a  specified  mean  of  30  minutes )  and  95th 
percentile  (where  the  maximum  value  of  the  95th  percentile,J4iax<^®  given 
as  60  minutes )  and  Figure  B-8C  which  is  the  superimposition  of  Figure  B-8A 
on  Figure  B-8B. 

Therefore,  tests  performed  to  demonstrate  the  attainment  of  both  the 
percentiles  in  question  actually  demonstrates  the  attainment  of  values  of 
0  &  ^which  are  almost  identically  acceptable  to  a  dual  requirement  of  the 
Mean  and  l%ax*  follows  then  that  an  accept  decision  relative  to  both 
percentiles  would  also  approximately  signify  an  accept  decision  for  a  dual 
Mean  and  Mnax  requirement. 

Since  both  percentiles  can  be  considered  independent  for  practical 
purposes,  the  same  samples  can  be  used  for  demonstrating  both  percentiles, 
therefore,  if  desired  the  tests  may  be  run  simultaneously. 

B.90.3  Procedure  -  Sample  tasks  shall  be  selected  with  respect  to  the 
procedure  defined  for  variable  sample/sequential  tests.  The  same  sample 
tasks  may  be  used  simultaneously  in  the  demonstration  of  both  the  Mean 
and  Mjj,^  requirements.  Table  1*,  Table  2*,  Table  3*  (which  are  based  upon 
the  sequential  probability  ratio  test  of  proportion)  define  the  accept/ 
reject  criteria  for  the  values  of  the  required  mean,  M^ay  (when  defined 

as  the  maximum  90th  percentile  value),  Mmav  (when  defined  as  the  maximum  95th 
percentile  value),  respectively.  The  number  of  observations  greater  than 
and  less  than  the  required  values  of  the  Mean  and  l%ax  shall  be  cumulated 
separately  and  compared  to  the  decision  values  shown  in  the  tables  applicable 
to  the  two  requirements.  When  one  plan  provides  an  accept  decision,  attention 
to  that  plan  shall  be  discontinued.  The  second  plan  shall  continue  until 
a  decision  is  reached.  The  equipment  shall  be  rejected  when  a  decision 
to  reject  on  either  plan  has  occirrred  regardless  of  the  status  of  the  other 
plan.  The  equipment  shall  be  accepted  only  when  an  accept  decision  has  been 
reached  on  both  plans.  If  no  accept  or  reject  decision  has  been  made  after 
100  observations,  the  following  rule  shall  apply: 


NOTE:  *Tables  1,  2  &  3  are  appropriate  to  Test  Plans  A, ,  B,  and  B2, 
respectively. 
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Plan  A]_  -  Accept  only  if  29  or  less  observations  are  more  than 
the  value  of  the  required  Mean. 

Plan  Bq  -  Accept  only  if  5  or  less  observations  are  more  than  ^naxc’ 

Plan  B2  -  Accept  only  if  2  or  less  observations  are  more  than  Ifciaxc* 

It  is  recognized  and  accepted  that  truncation  will  somewhat  modify 
probability  of  acceptance  characteristics  as  described  in  the  following 
section. 


B.90.4  The  PC  Curve  -  The  operating  characteristic  curve  for  the  test 
procedure  may  be  determined  by  mapping  the  probability  of  acceptance  for 
various  selected  points  on  a  diagram  of  the  acceptable  and  unacceptable 
regions  such  as  Figure  B-8D.  (Note  that  any  point  can  be  identified 
uniquely  by  the  coefficient  of  Q,  where  Q  =  In  (required  Mean),  on  the 
ordinate  and  the  coefficient  of  /q  on  the  abcissa  -  let  the  coefficient  of 
Q  be  denoted  as  (C)  and  the  coefficient  of  be  denoted  as  (K)  -  for 
example,  point  B  on  Figure  B-8D  can  be  uniquely  located  at  C  =  3/4,  K  =  .4). 
Each  point  is  also  representative  of  a  particular  lognormal  distribution 
possessing  unique  percentiles  for  the  values  given  for  (required  maximum 
value  for  Mean)  and  M^fly.  respectively. 


The  probability  of  acceptance  relative  to  any  point  is  equal  to  the 
compound  probability  of  passing  the  percentile  test  relative  to 
(Test  Al)  and  passing  the  percentile  test  relative  to  ^nax  (Test  B1  or  B2). 

Let  Pgi,  Pg2  be  the  probability  of  passing  test  Ai,  Bq,  B2, 

respectively  for  any  given  unique  combination  of  0  and  cf  (a  particular  point). 


PA1>  PB1>  PB2  niay  be  determined  by  calculating  Y^l,  Ybi,  YB2  from  the 
following  equations: 


„  _  JQ  (1-C) 

^A1  ic 


(7-1) 


-CQ 


(7-2) 


and  entering  Figure  B-8E  (for  Test  Al)  with  the  calculated  value  of 

and  Figure  B-8F  (for  Test  Bi)  or  Figure  B-8G  (for  Test  B2)  with  the 
calculated  value  of  Yg]^  or  Yg2.  The  corresponding  value  of  probability  of 

acceptance,  Pj^,  and  Pbi  or  Pb2  (whichever  of  the  B  tests  are  appropriate) 
is  read  from  each  Figure  and  Pat  and  the  appropriate  Pbi  ^62  value  are 
multiplied.  Tne  result  of  this  multiplication  is  the  probability  of 
acceptance  of  a  unit  having  a  particular  0  and  d  characteristic  defined 
by  (C)  and  (K). 
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Repeatir^  the  above  for  a  number  of  points  as  in  Figure  B-8H 
defines  an  operating  characteristic  map  relative  to  a  given  dual 
requirement.  Note  that  probabilities  of  acceptance  always  decrease 
as  the  point  is  located  upward  or  to  the  right  and  always  increase 
as  the  point  in  consideration  is  located  downward  or  to  the  left  on  the 
figure.  Hence,  sufficient  knowledge  of  test  characteristics  can  be  gen¬ 
erated  by  evaluating  relatively  few  points. 
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Figure  B-8A 
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Figure 


Figure  B-8C 
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PROBABILITY  OF  ACCEPTANCE 


PROBABILITY  OF  ACCEPTANCE  PB 


Figure  B-8H 
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TEST  FOR  HEAR  MAIIITIIIABCI  TINE  (CORRECTIVE 
PREVERTATIVE  (JOMBIllAftON  OF  CORBBCTIVt  AHD  PREVElTPAttVE)  AHD 
BiTu: — u^harai"'-  'inig  BeTOden'B'ippTicaBie  to'  "aanmi'Htratioir  or  'tire — 
following  indices  of  maintainability:  Mean  Corrective  Maintenance 
Time  (Pq),  Mean  Preventive  Maintenance  Time  Mean  Maintenance  Time 

(includes  preventive  and  corrective  maintenance  actions)  (Vip/g),  and 

Hnax  (percentile  of  repair  time). 

BlOO.l  Conditions  of  Use  -  The  procedures  of  this  method  for  demonstra¬ 
tion  of  Pq,  are  based  on  the  Central  Limit  Theorem.  No  information 
relative  to  the  variance  (d^)  of  maintenance  times  is  required.  It  may 
therefore  be  applied  whatever  the  form  of  the  underlying  distribution, 
provided  the  sample  size  is  adequate.  The  minimum  sanqple  size  is  set 
at  30.  The  actual  sample  size  (if  greater  than  30  are  required)  shall 
be  determined  for  each  equipment  to  be  demonstrated,  and  shall  be 
approved  by  the  procuring  activity. 

The  procedure  of  this  method  for  deii»nstrating  Mmav  valid  for 

those  cases  where  the  underlying  distribution  of  corrective  maintenance 
task  times  is  lognormal. 

B100.2  Quantitative  Requirements  -  Application  of  this  plan  reqxdres 
identification  of  the  index  or  indices  of  interest  and  specification  of 
quantitative  requirements  for  each.  When  dononstration  involves  H*  or 
or  a  combination  of  both,  consumer's  risks  will  be  specified.  When 
demonstration  involves  Mna vg .  the  percentile  point  which  defines  the 

specified  value  of  will  be  specified.  A  minimum  sample  size  of  30 

corrective  maintenance  tasks  is  required  for  d^ionstration  of  corrective 
maintenance  indices.  A  minimimi  san5)le  of  30  preventive  maintenance  tasks 
is  required  where  demonstration  of  preventive  maintenance  indices  by 
sampling  is  permitted,  and  is  to  be  accomplished  by  this  method. 

B100.3  Task  Selection  and  Performance  -  Sample  tasks  shall  be  selected 
in  accordance  with  the  procedure  outlined  in  Api)endix  "A.”  The  duration 
of  each  shall  be  recorded  and  used  to  ccmpute  the  following  statistics: 
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Xp/c 


—  f  V  J  f  Y 
-■^C  'C  '  Tp]7^  _»P|^ 


i 


'^'maxc  “  Antilog  n 

(Base  e)  ^  In  X^i 

"c 


Inhere  ’1'  3s  the  value  of  the  independent  variable  log-normal  function  v,hich 
corresponds  to  the  percentile  point  at  v>hich  ^^naxc  established. 

For  the  t\^o  most  common  percentile  points,  901  and  95%,yis  1.282  and  1.645 
respectively. 

B. 100.4  Accept/Re iect  Criteria  -  A  table  of  the  normal  distribution  function 
shall  be  consuilted  for  values  of  0  (for  a  one-tail  test)  ivtiich  corresponds  to 
the  specified  level  of  consumer  risk  p,  Tlae  follcn^dn^  table  nrovides  values 
of  vvhich  correspond  to  the  most  commonly  used  values  of  p. 

TABLE  V 

^  Vs.  ^ 


- P - 

0.84 

20% 

1.04 

15% 

1.28 

10% 

1.65 

5% 

T 


Accept/ reject  criteria  shall  be  computed  for  each  specified,  index  in  accordance 
with  the  folloi^ring  sections: 
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B. 100. 4.1  Test  for  Mean  Corrective  Maintenance  Time  (y^)  -  The  accept/reject 
value  for  is:  ^  ^ 

Accept  if  Ug  ( specified )  ^  /  j^dc 


=  Std.  deviation  of  sample  of 
corrective  maintenance  tasks. 


Reject  if  y„  (specified)  <  Xg  /  sz^d^ 


B. 100. 4. 2  Test 
value  for  Ppjj,  is 


cept /reject 


for_Mean  ^eventive  Maintenance  Time  (Upm)  “ 

Xpjj  /  deviation  of  preventive 


maintenance  tasks. 


Accept  if  ppjjj  (specified) 


Reject  if  ^pm  (specified)  Xp^  /  g^dpm 


‘pm 


B. 100.4. 3  Test  for  the  Mean  of  all  Maintenance  Actions  (Vp/c)  ~  The  accept/ 
reject  value  of  Up/c 


Ve’'*' 


M 


^pm  (^c  ^c)^  / 


c  ^  ^  pm  > 


^c  ^pm  (^c  ^ 


^/c  (specified)  Xp/^  ^ 


^p/c  (specified)  < 


^pn  ( ^c  ^c )  ^  ( ^pm  ^pn  ^ 

V  ^c  ^^^pm  (^c  ^  ^pm^ 

(Tq  dg)  /  n^  (fpm  dpj) 


Accept 


Re j  ect 


n  n  (f  /  f  )' 
c  pm  c  pm 


B. 100.4. 4  Test  for  M^^^  -  The  accept/reject  value  for  M^a^^  is: 


M'max,  =  Antilog 


®  ( Base  e ) 


“ 

nc 

^  (inXci) 

i=l 

“1 

^c 

E  (inXci)^ 
i=l 

n<;  2  1 

-  (I  In  Xci)  \ 
i=l 

/'^l» 

1 

nc 

J 
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Test  Method  10 


TESTS  FOR  FERCEIfrilES  AND  MAINTENANCE 
TDE  (CORRECTIVE  PKEVEHTATIVE  MAINTENANCE) 


BllO.  General  -  Thi^  method  employs  a  test  of  proportion  to  demonstrate 
achievement  of  when  ^he  distribution  of 


corrective  and  preventive  maintenance  repair  times  is  unknown. 


BllO.l  Conditions  of  Use  -  This  method  is  intended  for  use  in  cases 
where  no  information  is  available  on  the  underlying  distribution  of 
maintenance  task  times.  The  plan  holds  the  confidence  level  at  75%  or 
90^  as  may  be  desired  and  requires  a  minimum  sample  size  (N)  of 
50  tasks. 


BllO. 2  Quantitat^e  I^uirements  -  Application  of  this  method  required 
specification  of  ^jj,  Hnaxct  percentile)  or  HnaXp^  (95th 

percentile)  and  selection  of  75%  or  90?  confidence  level. 

BllO. 3  Task  Selection  and  Performance  -  Sample  tasks  shall  be  selected 
in  accordance  with  the  proceudres  of  Appendix  "A."  The  duration  of  each 
task  will  be  compared  to  the  required  value( s )  of  the  specified  index 
or  indices  Hnaxct  ^axpn^  recorded  as  greater 

than  or  lesser  than  each  index. 


BllO. 4  Accept/Reject  Criteria  -  The  item  under  test  shall  be  accepted 
when  the  number  of  observed  task  times  which  exceed  the  required  value 
of  each  specified  index  is  less  than  or  equal  to  that  shown  in  the 
Table  (B-lOA  or  B-lOB)  corresponding  to  each  index  for  the  specified 
confidence  level. 


BllO. 4.1  Test  for  the  Median  -  Table  B-IQA  below  is  a  test  of  the  median 
for  corrective  and  preventive  maintenance  tasks.  The  acceptance  level 
is  shown  for  two  confidence  levels  and  a  sample  size  (N)  of  50  tasks. 
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Acceptance  Table 

Sample  size  =  50 

Confidence  Level 

1%  9055 

Acceptance  Level 
22  20 


BllOA.a  Test  for  Jfeaxc  ^axpm  "  Table  B-lOB  is  a  test  for 

and  J4naxpu  a.t  the  95th  percentile.  The  acceptance  level  is  shown  for 
two  conffence  levels  and  a  sample  size  (N)  of  50  tasks. 


Table  B-lOB 

Acceptance  Table  for  J^axct 
Sample  size  =  50 

Confidence  Level 

75*  90* 

Acceptance  Level 
1  0 


NOTE:  Reference  -  "Introduction  to  Statistical  Analysis"  by  Dixon  & 
Massey,  Page  230,  McGraw-Hill  Company,  2nd  Edition,  1957. 
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Test  Method  11 

TBST  PC®  ramirPATIVE- MAIHTBRAlfC®  TIMES 


B.120  General  -  This  method  provides  for  maintainability  demonstra¬ 
tion  when  the  specified  index  involves  Upjj  and/or  14naXpjj  &3id  when  all 

possible  preventive  maintenance  tasks  are  to  be  performed. 

B. 120.1  Conditions  of  use  -  All  possible  tasks  are  to  be  performed  and 
no  allowance  need  be  made  for  underlying  distribution. 

B.120.2  Quantitative  requirements  -  Application  of  this  plan  requires 
quantitative  specification  of  the  index  or  indices  of  interest.  In 
addition,  the  percentile  point  defining  J^iaxpin  stipulated 

when  li%axp[j  is  of  interest. 

B.120.3  Task^ selection  and  performance  -  All  preventive  maintenance 
tasks  will  be  performed.  The  total  population  of  PM  tasks  will  be 
defined  by  properly  weighing  each  task  in  accordance  with  relative 
frequency  of  occurrence  as  follows:  Select  the  particular  task  for 
which  the  equipment  operating  time  to  task  performance  is  greatest 
and  establish  that  time  as  the  reference  period.  Determine  the  fre¬ 
quency  of  occurrence  (fpgn)  of  all  other  tasks  during  the  reference  period. 
Where  the  frequency  of  occiirrence  of  a  given  task  is  a  fractional  nvun- 
ber,  the  frequency  shall  be  set  at  the  nearest  integer.  The  total 
population  of  tasks  consists  of  all  tasks  with  each  repeated  in  accord¬ 
ance  with  its  frequency  of  occurrence  during  the  reference  period. 

B. 120. 4  Accept/reject  criteria  - 

B. 120. 4.1  Test  for  Ppjjj  -  the  mean  shall  be  computed  as  follows: 

^pmi  ^  ^^pmi  ^ 

Upjjj  (Actual)  =  — 


Where:  f-nmi  frequency  of  occurrence  of  the  i'*'^  task  in  the  refer¬ 

ence  periM. 

k  is  the  number  of  different  PM  tasks. 

Efpjii  is  the  total  number  of  PM  tasks  in  the  population. 

Accept  if:  Ppgjj  ( required Ppgjj  (actual) 

Reject  if:  ( required )<  ppjjj  (actual) 
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B.iao.4.2  Test  for  "  The  PM  tasks  shall  be  ranked  by  magnitude 

(lowest  to  highest  value).  The  equipment  shall  be  accepted  if  the 
magnitude  of  the  task  time  at  the  percentile  of  interest  is  equal 
to.  or  less  than  the  required  value  of  Mmaxpjjj* 

U.S.  GOVERNMENT  PRINTING  Ori  ICE:  1984-705-040/a4158 
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DEMONSTRATION  AND  EVALUATION  OF  EQUIPMENT/SYSTEM  BUILT- IN-TEST/EXTERNAL 
TEST/FAULT  ISOLATION/TESTABILITY  ATTRIBUTES  AND  REQUIREMENTS 

1 .  SCOPE 

1.1  Purpose:  This  addendum  to  MIL-STD-471A  provides  procedures  for  the 
evaluation  and  demonstration  of  equipment/ system  Built-in-Test  (BIT)  and 
external  test  subsystem  Fault  Isolation  and  Testabi 1 ity  attributes  which 
relate  to  maintainability  and  various  logistic  support  factors  which  are 
impacted  by  maintainability.  Its  purpose  is  to  supplement  the  more  conventional 
maintainability  test  requirements  (which  deal  with  accessibility,  time,  and 
human  factors)  with  tests  appropriate  to  Built-in-Test,  External  Test,  and 
Fault  Isolation  capabilities  of  the  system  or  subsystem  in  question. 

1.2  Application:  This  addendum  is  intended  for  use  when  evaluation  and 
demonstration  of  BIT  and  external  test  subsystem  (tester)  Fault  Isolation, 
and  Testability  attributes  and  requirements  for  hardware  procurement  is 
required.  It  provides  evaluation  and  demonstration  procedures  for  use  at 

the  equipment/system  Operational  (organizational)  Level,  at  the  Shop  Maintenance 
Level  and  the  Depot  Maintenance  Level.  Such  demonstrations  shall  be  called 
out  as  a  separate  part  of  the  maintainability  program  and  may  be  performed 
independently  of  the  tests  called  out  in  other  portions  of  MIL- STD-471.  Such 
demonstrations  shall  generally  precede  conventional  maintainability  demonstra¬ 
tions  contained  in  other  portions  of  MIL-STD-471A. 

2.0  DEFINITIONS 

Initial  Isolation:  Isolation  to  the  equipment/system  subunit  which  must 
be  replaced  on  line  to  return  the  equipment/ system  to  operation.  A  subunit 
can  be  a  modular  assembly,  a  printed  circuit  card  which  is  part  of  a  non¬ 
removable  drawer,  or  a  component  such  as  a  crystal  or  antenna  subsection. 

In  the  event  that  the  maintenance  concept  requires  a  subunit  to  be  removed. 
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repaired  and  then  replaced  in  the  equipment/system,  initial  isolation  includes 
both  isolation  to  the  failed  subunit  and  isolation  to  the  failed  and  removable 
portion  of  the  subunit. 

Initial  Isolation  Level  of  Ambiguity:  The  number  of  possible  equipment/ 
system  subunits,  as  defined  above,  identified  by  the  Built-in-Test,  external 
test  equipment,  or  manual  test  procedure,  which  might  contain  the  failed 
component.  It  is  possible  that  a  combination  of  Built-in-Test,  external 
special  purpose  test  equipment,  and  manual  procedures  may  be  necessary  to 
effect  isolation.  For  example,  if  an  equipment  test  subsystem  (Built-in, 
external,  manual)  isolates  a  fault  to  one  of  two  subunits,  the  level  of  ambiguity 
is  equal  to  two;  if  it  isolates  it  to  one  of  three  subunits  the  level  of 
ambiguity  is  equal  to  three. 

Secondary  Isolation:  Isolation  to  the  subunit  component/part  which  must 
be  replaced  in  the  shop  to  return  the  subunit  to  serviceable  condition.  The 
subunit  component/part  can  be  a  modular  assembly,  a  printed  circuit  card,  or  a 
piece  part.  In  the  event  that  Initial  Isolation  Level  of  Ambiguity  necessitates 
the  removal  and  replacement  of  two  or  more  subunits  (only  one  of  which  has 
malfunctioned)  Secondary  Isolation  includes  both  isolation  to  the  actual  failed 
subunit  (after  removal  from  the  system/equipment)  and  isolation  to  the  component/ 
part  in  the  failed  subunit. 

Secondary  Isolation  Level  of  Ambiguity:  The  number  of  possible  subunit 
components/parts  as  defined  above,  identified  by  Built-in-Test,  external 
testers  or  manual  test  procedures,  which  might  contain  the  failed  component/ 
part  (it  is  possible  that  a  combination  of  Built-in-Test,  external  testers,  and 
manual  procedures  might  be  necessary  to  effect  isolation).  For  example,  if  a 
subunit  test  system  (say  a  tester)  isolates  the  faulty  component  to  one  of  two 
printed  circuit  cards,  the  level  of  ambiguity  is  two,  if  it  isolates  to  one  of 
three  printed  circuit  cards  the  level  of  ambiguity  is  equal  to  three. 
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Tertiary  Isolation;  Isolation  (usually  at  the  depot  level)  at 
the  lowest  replacement  level,  of  a  subunit  or  a  component/part  belonging 
to  a  subunit  to  return  the  item  in  question  to  serviceable  condition.  The 
lowest  replacement  level  may  constitute  a  modular  assembly,  printed 
circuit  card,  or  a  piece  part. 

Tertiary  Isolation  Level  of  Ambiguity:  The  number  of  possible 
item  parts  (or  combinations  of  item  parts),  identified  by  testers, 
test  points,  or  manual  test  procedures,  which  must  be  replaced  (or 
individually  tested  and  then  replaced)  to  return  the  item  to  serviceable 
condition.  In  some  cases,  a  combination  of  parts  (more  than  one  part) 
may  be  required  to  return  an  item  to  serviceable  condition.  This 
isolation  level  in  some, instances  may  have  to  be  evaluated  on  a  qualitative 
basis  as  high,  medium,  or  low  based  on  the  characteristics  of  the  item, 
the  general  test  instruments  required  and  the  effectiveness  of  the  special 
instruments  and  testers  available. 

Manual  Procedures:  Any  procedure  which  requires  (1)  measurements 
using  general  purpose  test  equipment,  or  (2)  a  series  (more  than  one) 
of  sequential  remove  and  replace  actions  on  subunits  (subunit  component/ 
parts),  (lowest  level  replacements  internal  to  subunit  component/parts) , 
some  of  which  are  non-failed,  in  order  to  diagnose  and  isolate  a  failed 
subunit  (subunit  component/part),  (lowest  level  replacement  internal 
to  a  subunit  component/part). 

3.  general  procedure  for  EVALUATION/DEMONSTRATION  OF  BIT/FAULT 
ISOLATION/TESTABILITY  ATTRIBUTES  AND  REQUIREMENTS:  NOTE:  Paragraph  A. 10.4 
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of  MIL-STD-471A  calls  for  selection  of  a  Sample  of  Maintenance  Tasks  (which 
are  to  be  simulated)  equal  to  four  times  that  required  for  the  actual  Maintain¬ 
ability  Demonstration.  Paragraph  A. 10. 5  of  MIL-STD-471A  calls  for  a  failure 
mode  selection  and  a  failure  modes  and  effects  analyses  for  all  sample  tasks. 

The  level  to  which  the  failure  mode  will  be  chosen  is  dependent  on  the  scope  of 
the  requirement  or  the  particular  evaluations  of  interest.  If,  for  example, 
the  requirement  pertained  only  to  Initial  Isolation  characteristics,  just  the 
different  failure  modes  associated  with  the  outputs  of  the  subunits  need  be 
considered  in  the  analysis.  If  the  requirement  pertains  to  both  Initial  and 
Secondary  Isolation  characteristics,  just  the  different  failure  modes  associated 
with  the  outputs  of  each  subunit's  components/parts  need  be  considered  in  the 
analysis.  If  the  requirement  pertains  to  Initial,  Secondary  and  Tertiary 
Isolation  characteristics,  then  the  failure  modes  associated  with  the  outputs 
of  the  lowest  possible  replaceable  parts  (at  the  tertiary  level)  need  be 
considered  in  the  analysis. 

Where  test  or  evaluation  is  to  be  performed  relative  to  the  quality  or 
effectiveness  of  the  fault  detection  and  location  capabilities  of  the  equipment/ 
system,  the  following  procedures  shall  apply: 

a.  Each  sample  simulated  fault  selected  in  accordance  with 
paragraph  A. 10.4,  A. 10. 5  and  B.10.4  of  MIL-STD-471A  and  contained  in  Column  11 
(see  paragraph  A.10.4.j)  shall  be  analyzed  to  determine  whether  or  not  a  clear 
indication  of  equipment/system  failure  is  provided  and  whether  or  not  such 
indication  occurs  in  an  obvious  fashion.  This  procedure  does  not  require  a 
demonstration  of  repair  of  the  faults  induced. 

b.  Each  simulated  fault  shall  be  analyzed  to  determine  the  level  of 
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ambiguity  which  the  equipment/system  Built-in-Test,  external  test 
equipment,  or  manual  test  procedures  (documented  or  undocumented  in 
the  Technical  Orders)  performs  the  initial  isolation.  Where  appropriate, 
the  above  analysis  shall  be  performed  for  Secondary  and  Tertiary  levels 
of  isolation  during  the  test  phase.  The  results  of  the  above  shall  be 
depicted  in  the  form  of  an  Equipment/System  Testability  Profile  -  See 
Table  3. 

3.1  Development  of  Equipment/ System  Testability  Profile:  The  results 
of  the  fault  analysis  shall  be  presented  to  the  level  necessitated  by 
the  requirement(s) ,  following  the  format  of  Table  3. 

a.  Column  (1)  identifies  the  number  of  the  failure  simulated. 

b.  Column  (2a)  denotes  that  the  simulated  failure  is  immediately 
identified  as  causing  an  equipment/ system  failure. 

c.  Column  (2b)  denotes  that  the  simulated  failure  produced 
manifestations  out  of  the  ordinary,  but  which  could  not  immediately  be 
identified  as  indicative  of  equipment/ system  failure. 

d.  Column  (2c)  denotes  that  the  simulated  failure  was  not  detected. 

e.  Column  (3a)  denotes  that  Initial  Isolation  to  the  failed  subunit 
was  accomplished  at  least  partially  by  Built-in-Test. 

f.  Column  (3b)  denotes  that  Initial  Isolation  to  the  failed  subunit 
was  accomplished  at  least  partially  by  external  test  subystems. 

g.  Column  (3c)  denotes  that  Initial  Isolation  to  the  failed  subunit 
was  accompished  at  least  partially  by  manual  test  procedures,  either 
documented  in  the  Technical  Orders  or  not  contained  in  the  Technical 
Orders . 
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N0TE#1 :  It  is  possible  that  for  any  given  isolation,  two  or  more 
of  the  techniques  discussed  above  (e,  f,  g)  must  be  used. 

h.  Column  (3d)  denotes  the  ambiguity  level  associated  with  the 
fai 1 ure. 

N0TE#2:  The  isolation  profile  for  Initial,  (Secondary)^  (Tertiary) 
Isolation  shall  be  if  possible  broken  down  as  in  the 
circled  portions  of  Table  3A  and  as  per  the  following 
example: 

LEVEL 

FAILURE  NO.  BUILT- IN-TEST  EXTERNAL  TEST  MANUAL  PROCEDURES  AMBIGUITY 

1  X  4 

1  X  2 

1  X  1 

In  this  case,  for  failure  number  1  -  the  Built-in-Test  is  capable  of 
isolation  to  four  subunits.  External  test  subsystems  are  used  to  reduce 
the  ambiguity  further  to  one  of  two  subunits.  Manual  test  procedures  are 
required  to  isolate  to  a  single  failed  unit. 

i.  Column  (4a)  denotes  that  Secondary  Isolation  to  the  failed  subunit 
component  or  part  was  carried  out  at  least  partially  with  the  aid  of  the 
equipment/system  Built-in-Test  or  with  Built-in-Test  associated  with  the 
failed  unit  itself. 

j.  Column  (4b)  denotes  that  Secondary  Isolation  to  the  failed  sub¬ 
unit  component  or  part  was  carried  out  at  least  partially  with  the  aid  of 
external  testers  (card  testers,  special  test  subystems).' 

k.  Column  (4c)  denotes  that  Secondary  Isolation  to  the  failed 
subunit  component  or  part  was  carried  out  at  least  partially  by  manual 
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means  either  documented  in  the  Technical  Orders,  or  not  documented 
in  the  Technical  Orders. 

l.  Colimin  (4d)  denotes  the  ambiguity  level  associated  with  the 
failure.  The  isolation  table  shall  if  possible  be  broken  down  as  in 
Note  #2  and  its  example. 

m.  Column  (5a)  denotes  that  Teritary  (depot)  Isolation  of  the 
failed  item(s)  was  accomplished  at  least  partially  by  Built-in-Test 
associated  with  the  item. 

n.  Column  (5b)  denotes  that  Tertiary  Isolation  of  the  failed 
item(s)  was  accomplished  at  least  partially  by  special  test  systems 
(testers) . 

0.  Column  (5c)  denotes  that  Tertiary  Isolation  of  the  failed 
item(s)  was  accomplished  at  least  partially  by  manual  means,  either 
documented  in  the  Technical  Orders  or  not  documented  in  the  Technical 
Orders. 

3. 2  Procedure  for  Development  of  Equipment  Evaluation  Profile  for 
Failure  Detection  and  Isolation. 

a.  For  columns  2a-2c  for  each  failure  simulated,  place  an  x  in 
the  appropriate  column. 

b.  For  column  3a,  4a,  and  5a  for  each  failure  simulated  (as  in 
table  3),  place  an  x  in  the  column  if  Built-in-Test  was  used  for 
Initial,  (Secondary ),(Terti ary)  Isolation  and  indicate  in  corresponding 
column  3d,(4d)  ,(5d) ,  the  ambiguity  level  resulting  from  the  use  of 
Built-in-Test  (if  Bui  It- in- Test  was  not,  or  could  not  be  used  in  the 


84 


MIL-STD-471A 
8  December  1978 


Initial j (Secondary),  (Tertiary)  Isolation  process,  leave  that  part  of 
column  3a, (4a),  (5a)  blank  and  leave  the  corresponding  part  of  column  3d, 
(4d),  (5d)  blank). 

c.  For  columns  3b,  4b,  and  5b  for  each  failure  simulated  (as  in 
Table  3),  pla^e  an  x  in  the  column  if  external  test  systems  or  testers 
were  used  for  Initial, (Secondary), (Tertiary)  Isolation  and  indicate  in 
the  corresponding  column  3d,(4d),  (5d)  the  ambiguity  level  resulting 
from  the  use  of  external  test  systems  or  testers  (if  external  test 
systems  or  testers  were  not,  or  could  not  be  used  in  the  Initial; 
(Secondary)^  (Tertiary)  Isolation  process,  leave  that  part  of  column  3b, 
(4b), (5b)  blank  and  leave  the  corresponding  part  of  column  3d,(4d);  (5d) 
blank). 

d.  For  columns  3c,  4c,  and  5c  for  each  failure  simulated  (as  in 
Table  3)  place  an  x  in  the  column  if  manual  means  using  general  purpose 
test  equipment  and  procedures  documented  in  the  Technical  Orders  were  used 
in  the  Initial,  (Secondary),  (Tertiary)  Isolation  process.  Place  a  zero  (o) 
in  the  column  if  manual  means  using  general  purpose  test  equipment  and 
procedures  not  documented  in  the  Technical  Orders  were  used  to  perform 
Initial,  (Secondary),  (Tertiary)  Isolation.  It  is  possible  to  have  both  an 
X  and  a  zero  (o)  entry  in  the  same  part  of  the  column  (which  denotes  that 
both  documented  and  undocumented  procedures  were  necessary  to  effect 
isolation).  The  corresponding  column  3d, (4d),  (5d)  should  be  made  to 
indicate  the  resulting  ambiguity  level. 

e.  For  columns  3d  and  4d,  as  described  above. 
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f.  For  column  5d,  indicate  in  the  corresponding  place  in  column  5d 
the  ambiguity  level  resulting  (high,  medium,  low)  from  each  entry  in 
5a,  5b  and  5c. 

4.  USE  OF  THE  EQUIPMENT  EVALUATION  PROFILE  FOR  EVALUATION  AND  TEST  PURPOSES: 
Using  the  information  compiled  on  the  profile,  data  can  be  computed  pertinent 
to  the  following  testability  attributes  (Figures  of  Merit)  which  can  be  used 
either  for  evaluation  or  demonstration  purposes. 

=  Proportion  of  Sample  Failures  detected. 

Pg^  =  number  of  x's  in  column  2a 

total  number  of  failures  simulated 

For  example  Pg^  =  .8  would  indicate  that  80%  of  the  sample  failures  were 
detected. 

P-IB^ »  (^28^  ^  ^  “  Pf'oportion  of  sample  failures  for  which  Built- 

in-Test  (BIT)'  and/or  special  purpose  external  test  systems  (SPETS)  are 
effect! ve'to'reduce  ambiguity  for  Initial  isolation,  (Secondary  Isolation), 
(Tertiary  Isolation). 

■  number  of  failures  for  which  BIT  and/or  SPETS 
®  reduce  Initial,  (Secondary),  (Tertiary)  Isolation 

Ambi  gui  ty _ 

Total  Number  of  Failures  Simulated 

For  example,  P-jg^  =  .9  indicates  that  for  90%  of  all  sample  failures 

either  BIT  or  SPETS  or  BOTH  IN  COMBINATION  were  effective  in  reducing 

ambiguity  during  Initial  Isolation. 

*-la’  (^2a)’  (*-3a)  “  Average  resulting  ambiguity  level  associated  with 
;  the  use  of  Builtrin-Test  (BIT)  and/or  special 

;■  V  ,  purpose  external  test  systems  (SPETS)  for  Initial 
Isolation,  (Secondary  Isolation),  (Tertiary 
Isolation). 
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‘■la’  ^‘■2a^’  ^'■3a^  " 


where 

L.,-  =  lowest  value  of  ambiguity  associated  with  the  use  of  BIT 

a  I 

and/or  SPETS  for  the  i^*^  failure  in  order  to  effect  Initial,  (Secondary), 
(Tertiary)  Isolation. 

S  =  Total  number  of  failures  which  employ  either  BIT  or  SPETS  or 
both  to  effect  Initial , (Secondary) ,  (Tertiary)  Isolation. 

For  example  an  L-jg  =  3  in  conjunction  with  a  P-j^g  =  .9  would  indicate 
that  for  90%  of  all  failures  BIT  and/or  SPETS  are  capable  of  reducing 
ambiguity,  and  when  this  occurs,  the  associated  average  level  of 
ambiguity  =  3. 

Pal»  ”  Proportion  of  sample  failures  which  require 

“  only  the  use  of  Built-in-Test  (BIT)  for 

Initial  Isolation, (Secondary  Isolation), (Tertiary 
Isolation)  to  a  qi yen  level  of  ambiguity  or  less. 

P  1.  (P  0)5  (P  o)  =  #  of  failures  requiring  only  BIT  to  perform 
®  Initial,  (Secondary! , (Tertiary)  Isolation  to  a 

level  of  ambiguity  £  L _ 

Total  number  of  failures  simulated 

For  example,  P^^  =  .6  and  L  =  3  would  indicate  (for  Initial  Isolation) 

that  for  60%  of  all  failures,  Built-in-Test  is  capable  of  isolating 

failures  to  one  of  three  subunits  or  better. 

^hl ’  (^63^  "  Proportion  of  sample  failures  which  requires  only 

the  use  of  special  purpose  test  systems  (SPETS) 
for  Initial  Isolation,  (Secondary  Isolation), 
(Tertiary  Isolation)  to  a  given  level  of  ambiguity 
or  less.. 

=  #  of  failures  which  require  only  SPETS  for  Initial, 
(Secondary);  (Tertiary)  Isolation  to  ambiguity 
level  ^  L. _ 

Total  number  of  failures  simulated 
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For  example  a  =  .5  for  an  L=1  would  indicate  (for  a  subunit  comprised 
of  printed  circuit  cards)  that  only  half  of  the  failures  could  be  isolated 
to  a  single  printed  board  by  the  special  testers  provided. 

*^cl  ’  ^^c3^  "  Proportion  of  sample  failures  which  require 

some  degree  of  Manual  Testing  (documented  in 
Technical  Orders)  to  effect  Initial  Isolation, 
(secondary  Isolation),  (Tertiary  Isolation). 

^cl  ’  ^^02^’  =  number  of  x's  in  column  3c,  (4c),  (5c) 

Total  number  of  failures  simulated 

For  example  Pj.-]  =  .2  would  indicate  that  20%  of  all  sample  failures 

required  the  application  of  manual  test  procedures  (documented  in  the 

Technical  Orders)  in  order  to  effect  Initial  Isolation. 

^dl  ’  ^^d3^  "  Proportion  of  sample  failures  which  require 

some  degree  of  Manual  Testing  (NOT  documented  in 
Technical  Orders)  to  effect  Ini tial  Isolation, 
(Secondary  Isolation),  (Tertiary  Isolation). 

Pdi .  (Pd2^  ^^d3^  =  number  of  zeros  in  column  3c,  (4c)^  (5c) 

Total  number  of  failures  simulated 

For  example  P^j-]  =  .1  would  indicate  that  10%  of  all  sample  failures 

required  the  application  of  manual  test  procedures,  not  documented  in 


the  Technical  Orders,  in  order  to  effect  Initial  Isolation. 

^el’  ^^e2^^^e3^  "  Proportion  of  sample  failures  for  which  Built-in- 

Test  (BIT)  and/or  special  purpose  external  test 
systems  (SPETS)  are  capable  of. effecting  Initial 
Isolation^ (Secondary  Isolation) ^ (Tertiary  Isolation) 
to  a  gi yen  level  of  ambiguity  or  better. 

^el  ^^e2^’  ^^e3^“  ^  failures  requiring  either  BIT  or  SPETS  or  both 
’  for  Initial, (Secondary), (Tertiary)  Isolation  to 

ambiguity  level  £  L 

Total  number  of  failures  simulated 

For  example  a  Pel  =  .6  and  an  L  =  2  would  indicate  that  for  60%  of  all 
failures,  BIT  or  SPETS  or  any  combination  thereof,  is  capable  of 
effecting  Initial  Isolation  to  one  of  two  subunits  or  better. 
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§ 


p  (Pp?)>  (PpS^  ~  Proportion  of  sample  failures  for  which  Built-in- 

Test  is  effective  in  Initial  Isolation,  (Secondary 
Isolation),  tertiary  Isolation)  in  reducing 
ambiguity. 

Ppi  ’  (Ppp)»  (^P3)  ~  _ _ Hi _ 

Total  number  of  failures  simulated 

where 


m  =  Number  of  simulated  failures  in  which  Initial,  (Secondary), 
(Tertiary)  Isolation  takes  place  (ambiguity  is  reduced) 
at  least  partially  through  the  use  of  Built-in-Test.  = 
number  of  x's  in  column  3a,  (4a),  (5a). 

For  example  a  Pp-j  =  .7  indicates  that  Built-in-Test  is  effective  to 
some  extent  in  removing  ambiguity  during  Initial  Isolation  for  70%  of  all 
failures. 

^al  ’  "  Proportion  of  sample  failures  for  which  special 

^  ^  ^  purpose  external  test  systems  are  effective  in 

Initial  Isolation, (Secondary  Isolation), (Tertiary 
Isolation)  in  reducing  ambiguity. 

"gi’  <V’’  ''’g3>  _ 

Total  number  of  failures  simulated 

where 

K  =  Number  of  simulated  failures  in  which  Initial,  (Secondary), 
(Tertiary)  Isolation  takes  place  (ambiguity  reduced)  at 
least  partially  through  the  use  of  special  purpose  external 
test  systems. =  number  of  x's  in  column  3b,  (4b),  (5b). 

For  example  a  Pg2  =  -8  applied  to  a  subunit  comprised  of  printed  circuit 
cards  would  indicate  that  the  special  purpose  external  test  system  (say 
a  TESTER)  is  effective  to  some  extent  in  removing  ambiguity  during 
Secondary  Isolation  for  80%  of  all  failures. 
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*■61’  ^*‘52^’  “  Average  resulting  ambiguity  level  for  Built-in- 

Test  when  used  for  Initial  Isolation,  (Secondary 
Isolation),  (Tertiary  Isolation). 

•-bl  ’  ^‘-62^’  ^*-b3^  "  "I?  , 

^  ‘-bi 
1=1 

m 

m  =  Number  of  simulated  failures  in  which  Initial,  (Secondary), 
(Tertiary)  Isolation  takes  place  (ambiguity  is  reduced)  at 
least  partially  through  the  use  of  Built-in-Test.  =  number 


of  x's  in  column  3a,  (4a),  (5a). 

L|3j  =  Ambiguity  level  associated  with  the  ith  simulated  failure  after 
using  Built-in-Test  at  least  partialT^to  effect  Initial, 
(Secondary),  (Tertiary)  Isolation. 

This  indicates  for  example  in  the  case  of  Initial  Isolation,  that 

the  functional  Built-in-Test  features  of  the  equipment  will  on  the  average 

isolate  to  Ljji  different  subunits.  Taken  in  conjunction  with  the 

* 

Ppi  “  indicates  that:  For  80%  of  all  failures  Built-in-Test 

performs  isolation  and  when  it  does,  its  average  level  =  Ljji. 

*-cl ’  (*-c2^’  ^*-c3^  “  Average  resulting  ambiguity  level  for  special 

purpose  external  test  systems  when  used  for 
Initial  Isolation,  (Secondary  Isolation), 

(Tertiary  Isolation). 


=  Ambiguity  level  associated  with  the  ith  simulated  failure 
after  using  external  special  purple  test  systems  or  testers 
to  effect  Initial,  (Secondary),  (|T|ertiary)  Isolation. 

K  =  Number  of  simulated  failures  in  which  Initial,  (Secondary), 

(Tertiary),  Isolation  takes  place  at  least  partially  through 

the  use  of  external  special  purpose  test  systems  or  testers.  = 

1 

number  of  x's  in  column  3b,  (4b),  (^b). 
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For  example,  an  L^2  ~  ^  related  to  Secondary  Isolation  would  indicate 
that  on  the  average  when  special  purpose  test  equipment  or  testers  are 
used  in  Secondary  Isolation,  they  on  the  average  perform  isolation  down 
to  a  level  of  ambiguity  =  2.  Taken  in  conjunction  with  a  Pg2  =  -8 
it  indicates  that  80%  of  all  Secondary  Isolation  is  performable  with 
special  purpose  test  equipment  and  on  the  average  that  test  equipment 
isolates  to  ambiguity  level  =  2. 

5.  CONFIDENCE  LEVELS  FOR  TESTABILITY  ATTRIBUTES:  The  parameters  dealing 
with  proportion  (P^iJ  P^] >  etc.)  as  described  in  the  previous  paragraph 
represent  observed  estimates,  based  upon  sampling  of  the  actual  pro¬ 
portions  in  question.  For  evaluation  purposes,  it  is  usually  necessary 
to  define  the  probable  range  of  the  actual  equipment/ system  proportions 
from  the  results  of  sampling.  The  following  procedure  defines  an  approach 
to  this  end. 

PULC  =  P±Z,/P(1^ 

''  w 

PuLC  ~  The  upper  and  lower  bounds  of  the  actual  value  of  proportion 
for  a  given  confidence  interval. 

P  =  The  observed  estimate  of  proportion  from  the  sample. 

W  =  The  sample  size  from  which  P  is  computed. 

Z(.  =  A  co-efficient  dependent  on  the  confidence  interval  desired. 

The  following  table  provides  appropriate  values  of  for  various 
confidence  intervals: 


Ic 

Confidence  Interval 

1.28 

80% 

1.65 

90% 

1.96 

95% 
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5.1  Treatment  of  the  Values  of  L  (i.e.,  L]a,  L|^i,  ,  etc.) 

For  evaluation  purposes,  values  of  L  will  be  treated  as  point  estimates 
as  in  paragraph  4. 

6.  DEMONSTRATION  FOR  TESTABILITY  ATTRIBUTES  (VALUES  OF  P):  For  those 
instances  where  the  parameters  dealing  with  proportion  (Pia»  ^bl »  ^cl * 
as  described  in  paragraph  4,  have  been  levied  as  specification  requirements 
and  must  be  demonstrated,  the  following  procedures  will  apply: 

-  Case  (A)  for  situations  where  the  higher  the  value  cf  P,  the  better 

the  Testability  of  the  equipment/system. 

(i.e.,  P],  p1  ,  P  ,,  P^,,  etc.) 
a  2b  al  bl 

Accept  if 

P  >  P.  -  1.28J  P(l-P) 

\l  W 

Reject  otherwise 

-  Case  (B)  for  situations  where  the  lower  the  value  of  P,  the  better 
the  Testability  of  the  equipment/ system,  i.e.,  P^-i,  Pj.| ,  etc. 

Accept  if 

P  <  Pc  +  1.28,  I  P(l-P) 

^  W 

Reject  otherwise 

These  tests  provide  the  producer  with  a  10%  risk  of  rejection  (1.28  is  the 
coefficient  of  the  normal  distribution  which  yields  this  risk)  and  at  the 
same  time  provide  the  consumer  assurance  that  testability  designs  with 
significant  deviations  from  specified  values  will  be  rejected. 

Where  P  =  measured  testability  proportion  from  sample. 

Pg  =  specified  proportion. 

W  =  the  sample  size  f’^om  which  P  is  computed. 
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6.1  Demonstration  for  Values  of  L  (i.e.,  L-jQ,  L^^i,  L^-],  etc,): 

For  demonstration  purposes,  values  of  L  will  be  as  estimated  in 
paragraph  4. 

6.2  Validation  of  Data  used  for  Evaluation/Demonstration: 

In  order  to  validate  the  accuracy  of  the  data  contained  in  the  testability 
profile  and  hence,  determine  its  acceptability,  the  following  procedure  will 
apply: 

a.  The  observed  diagnostic  and  fault  isolation  characteristics  of  the 
sample  population  of  failures  induced  for  maintainability  demonstration  (see 
paragraph  A. 10. 4. Ik  of  MIL-STD-471A)  shall  be  documented  in  the  format  of 
Table  3. 

b.  The  data  above  shall  be  compared  with  the  data  related  to  the 
same  simulated  faults  contained  in  the  testability  profile. 

c.  If  90%  or  more  of  the  simulated  faults  used  in  the  maintainability 
demonstration  show  the  same  profile  results  as  depicted  in  the  testability 
profile  for  those  same  faults  the  profile  is  considered  acceptable. 

In  the  event  that  the  testability  profile  was  generated  totally  from 
actual  hardware  tests  in  the  presence  of  representatives  of  the  procuring 
activity,  the  profile  will  be  considered  valid  without  recourse  to  the  above. 

7.  FALSE  ALARM  RATE  EVALUATION/DEMONSTRATION:  False  Alarm  Rate,  defined  in 
terms  of  average  number  of  false  alarms/24  hour  period  of  equipment  or  system 
operating  time  (Xp),  may  be  evaluated  and  demonstrated  from  false  alarm  data 
resulting  from  controlled  tests  (i.e.,  reliability  demonstration  tests, 
operational  tests,  performance  tests)  subject  to  procuring  activity  surveillance. 
The  contractor  and  representatives  of  the  procuring  activity  shall  jointly 
determine  the  specific  data  sources  to  be  utilized.  The  contractor  shall 
prepare  a  plan  submitted  as  part  of  the  Maintainability  Demonstration  Plan 
(and  subject  to  Procuring  Activity  approval)  defining  the  procedures  to  be 
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utilized  in  the  collection  and  documentation  of  such  data. 

17.1  Procedures  Necessary  for  Demonstration  or  Evaluation;  A  cumulative 
^quipment/system  period  of  operating  time(s)  (T)  must  be  determined  over 
v^hich  the  demonstration  or  evaluation  will  take  place.  The  cumulative  period 
of  operating  time  (T)  must  as  a  minimum,  include  the  operating  time  duration 
of  the  reliability  demonstration  test(s).  When  the  evaluation  or  demonstration 
of  False  Alarm  Rate  is  to  be  terminated  prior  to  operational  field  use  testing, 
the  demonstration/evaluation  shall  be  considered  terminated  at  the  end  of 
contractor  reliability  demonstration. 

When  evaluation/demonstration  has  been  terminated,  the  following 
calculations  are  performed: 

A.  Let  expected  number  of  false  alarms  =  Xp^T 

where 

Xp5  =  the  specified  average  number  of  false  alarms/24  hour  period 
of  equipment  or  system  operating  time. 

T  =  the  cumulative  period  of  equipment  or  system  operating  time  over 
which  the  demonstration  or  evaluation  will  take  place. 

B.  Determine  from  data  available  the  total  nuniier  of  false  alarms  observed. 
7.1.1  For  Demonstration  Purposes:  Go  to  Figure  Bl-A  and  determine  the 
intercept  of  the  Expected  Number  of  False  Alarms  and  the  Actual  Number  of  False 
Alarms  Experienced. 

A.  If  the  intercept  lays  in  the  accept  range,  the  confidence  level  is 
approximately  75%  or  better  that  the  equipment  or  system  meets  or  betters  the 
False  Alarm  Rate  requirement  and  the  false  alarm  rate  is  deemed  satisfactory. 

B.  If  the  intercept  lays  in  the  reject  range  (or  if  at  any  time  during  the 
test  period  it  becomes  evident  that  rejection  will  occur)  the  false  alarm  rate 
is  deemed  unsatisfactory  and  the  equipment/system  is  rejected.  In  order  to  be 
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deemed  acceptable  after  rejection,  the  following  procedures  will  be  adhered  to: 

(1)  Analyses  of  the  fault  detection/isolation  system  or  unit 
must  be  performed  to  find  and  correct  the  design  deficiencies  which 
result  in  the  unacceptably  high  false  alarm  rate.  The  contractor  shall 
provide  the  procuring  activity  information  detailing  the  results  of  the 
above  analyses  and  the  steps  to  be  undertaken  to  correct  the  deficiencies, 

(2)  The  procuring  activity  may  then: 

(a)  Accept  the  equipment  on  the  condition  the  necessary 
design  changes  have  been  implemented  and  evidence  provided  of  such 
implementation,  or 

(b)  Require  a  new  limited  test  of  false  alarm  rate  upon 
implementation  of  the  necessary  design  changes  prior  to  final  acceptance 
(this  may  be  integrated  or  combined  with  final  equipment/ system  tests), 

7.1.2  For  Evaluation  Purposes:  Where  it  is  desired  to  estimate  from 

the  observed  data  the  actual  False  Alarm  Rate  to  a  75%  confidence  interval. 

Go  to  Figure  Bl-A  and  determine  the  intercept  point  of  the  boundary  line 
and  the  Observed  Number  of  False  Alarms.  Determine  the  corresponding 
Expected  Number  of  False  Alarms  value  at  that  point.  The  following 
relationship  will  provide  the  assessment  of  actual  false  alarm  rate  to 
the  confidence  desired: 

Ap  =  24( Expected  Number  False  Alarms  Value) 

where 

Xp  =  th(B  average  number  of  false  alarms/24  hour  period  of  operation. 

T  =  as  defined  previously. 
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